CN113127151B - Real-time copying method, system and medium for data in virtualization environment - Google Patents
Real-time copying method, system and medium for data in virtualization environment Download PDFInfo
- Publication number
- CN113127151B CN113127151B CN202110322237.1A CN202110322237A CN113127151B CN 113127151 B CN113127151 B CN 113127151B CN 202110322237 A CN202110322237 A CN 202110322237A CN 113127151 B CN113127151 B CN 113127151B
- Authority
- CN
- China
- Prior art keywords
- data
- virtual
- disk
- copying
- bitmap
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1464—Management of the backup or restore process for networked environments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1469—Backup restoration techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45595—Network integration; Enabling network access in virtual machine instances
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a real-time copying method of virtual environment data, which is applied to a server carrying a virtual environment and comprises the following steps: setting a virtual disk interface in a virtual environment; performing data copying operation, wherein the data copying operation comprises data copying operation between virtual environments and data copying operation between the virtual environments and the data receiving equipment; setting a file system backup mechanism, and performing data copying operation between virtual environments through the file system backup mechanism; setting a data merging mechanism, a temporary storage strategy and a bitmap copying mechanism, and performing data copying operation between a virtual environment and data receiving equipment through the data merging mechanism, the bitmap copying mechanism and a virtual disk interface; the bitmap copying mechanism is as follows: setting a plurality of bitmaps, marking the operation data through the bitmaps, and copying the operation data according to a marking result; the invention can continuously provide data protection in the backup process, saves resources in a bitmap mode and improves the performance of the disk.
Description
Technical Field
The present invention relates to the field of data replication, and in particular, to a method, system, and medium for real-time replication of data in a virtualized environment.
Background
With the popularization of virtualization technologies, more and more customer services are loaded on virtualization platforms, and currently, a virtualization backup is mainly to backup a hard disk of a Virtual Machine (VM), and data at a certain time point is backed up.
The existing virtualization backup mode mainly comprises the steps that backup software is in butt joint with a virtualization platform, snapshot operation is carried out on a VM when backup is started, and then a virtual hard disk file of the VM is transmitted to a backup server from the virtualization platform.
However, in this backup method, since the VM needs to be snapshot-operated at a specific time point, and the snapshot action itself has a large influence on the performance of the VM, it cannot provide continuous data protection, and only provides data protection according to the time point, and once a failure occurs, data in a period of time is lost.
In addition, the current HA scheme does not specifically optimize the VM environment, and data is still copied continuously at the VM operating system level, which causes a large system overhead of HA and a large influence on the system.
Disclosure of Invention
The invention mainly solves the problems that the current snapshot backup mode can not continuously provide data protection, data can be lost when a fault occurs, and the system can be influenced when HA backup is carried out.
In order to solve the technical problems, the invention adopts a technical scheme that: the method for copying the data of the virtual environment in real time is applied to a server carrying the virtual environment, and comprises the following steps:
setting a virtual disk interface in the virtual environment;
setting a file system backup mechanism, and performing data copying operation between virtual environments through the file system backup mechanism; the file system backup mechanism is as follows: setting a backup time point, wherein the virtual environment is backed up in a dirty page mode at the backup time point;
setting a data merging mechanism, a temporary storage strategy and a bitmap copying mechanism, and performing data copying operation between a virtual environment and data receiving equipment through the data merging mechanism, the bitmap copying mechanism and the virtual disk interface; the bitmap copying mechanism is as follows: setting a plurality of bitmaps, marking the operation data through the bitmaps, and copying the operation data according to a marking result.
Preferably, the bitmap copying mechanism further includes: dividing the space in the virtual disk into a plurality of disk spaces, and binding the disk spaces with the bitmap;
the operation data comprises read operation data of the virtual disk and write operation data of the virtual disk;
when the write operation data is carried out, recording a disk space for carrying out the write operation data as a first disk space, and marking a first bitmap by matching the first disk space with the first bitmap;
and when the write operation data is copied, copying the data in the first disk space corresponding to the first bitmap.
Preferably, the step of performing a data copy operation between a virtual environment and a data receiving device through the data merging mechanism, the bitmap copying mechanism, and the virtual disk interface further includes:
setting an IP address of a virtual disk interface;
the virtual disk interface copies the write operation data through the bitmap copying mechanism and the data merging mechanism;
and establishing communication with data receiving equipment through the IP address, a network card on the server and a network, and transmitting the write operation data to the data receiving equipment.
Preferably, the data merging mechanism is as follows: merging the write operation data, and recording the merged write operation data as first data;
and compressing the first data, packaging the first data into an IP packet and transmitting the IP packet to the data receiving equipment.
Preferably, the temporary storage policy is: setting a first physical space of a disk in the server;
when the data transmission between the virtual environment and the data receiving equipment is interrupted, storing the data of the write operation data into the first physical space, and simultaneously recording whether the data in the virtual disk changes or not by using a bitmap;
when the data transmission between the virtual environment and the data receiving equipment is recovered, writing the changed data of the virtual disk recorded by the bitmap into the first physical space, and transmitting the data in the first physical space to the data receiving equipment.
Preferably, the file system backup mechanism further comprises: sending an instruction to the server, wherein the server operates the virtual environment according to the instruction, and records the virtual environment which is operated as a first virtual environment;
the dirty pages in the memory of the first virtual environment are placed into a first virtual disk of the first virtual environment, snapshot operation is carried out on the first virtual disk at the backup time point, and the write operation data appearing after the backup time is written into a change file;
data copy operations between virtual environments are performed.
Preferably, the step of performing a data copy operation between virtual environments through the file system backup mechanism further includes: sending the data of the first virtual disk in the first virtual environment after the snapshot operation to a second virtual disk in a second virtual environment in the server to obtain the second virtual disk with the same data as the first virtual disk;
and the second virtual machine uses the mirror image of the first virtual machine, starts the second virtual machine when the first virtual machine is down, and uses the second virtual machine to execute the service of the first virtual machine.
The invention also provides a virtual environment data real-time copying system, which comprises: the system comprises a creating module, a file system backup module, a bitmap copying module and a data copying module;
the creating module is used for setting a virtual disk interface in a virtual environment;
the file system backup module is used for setting a backup time point, and performing backup on the virtual environment in a dirty page mode at the backup time point;
the bitmap copying module is used for setting a plurality of bitmaps, dividing the space in the virtual disk into a plurality of disk spaces, binding the disk spaces and the bitmaps, marking operation data through the bitmaps and copying the operation data according to a marking result;
the data copying module is used for performing data copying operation between virtual environments through the file system backup module;
the data copying module is used for performing data copying operation between the virtual environment and the data receiving equipment.
As a further improvement of the real-time virtual environment data replication system, the system further comprises a data merging module and a temporary storage module;
the data merging module is used for merging the write operation data, recording the merged write operation data as first data, compressing the first data, packaging the first data into an IP packet and transmitting the IP packet to the data receiving equipment;
the temporary storage module is used for setting a first physical space of a disk in the server;
the temporary storage module is used for storing the data of the write operation data into the first physical space when the data transmission between the virtual environment and the data receiving equipment is interrupted, and simultaneously recording whether the data of the virtual disk is changed or not by using a bitmap;
the temporary storage module is configured to, when data transmission between the virtual environment and the data receiving device is resumed, write the changed virtual disk data recorded by the bitmap into the first physical space, and transmit data in the first physical space to the data receiving device.
The present invention also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the computer program implements the steps of the virtual environment data real-time replication method.
The invention has the beneficial effects that:
1. the method for copying the data in the virtual environment in real time can realize the separation of the write operation from the virtual machine, and the data is transmitted through the network, so that the data can be copied in real time, and after the data is stored, the data of the virtual machine can be recovered to the time point of the fault or a certain previous time point in a data playback mode, and the accuracy of recovery in the fault is improved.
2. The real-time copying method of the virtual environment data can be used for data synchronization in HA dual-computer, the write data of the main virtual machine is further written into the hard disk of the standby machine (physical machine or VM) after being copied, and when the main machine fails, the data of the standby machine is up-to-date, so that the method can be used for starting application without data loss.
3. The virtual environment data real-time copying system can merge, compress and send data through the data merging module, improves the network transmission efficiency, and can reduce the probability of communication failure with external data receiving equipment through the temporary storage module.
4. The computer-readable storage medium can realize binding by setting the space of the bitmap and the virtual disk, only needs to copy once when copying, saves time and disk resources, deletes the bitmap after copying is finished, does not occupy the space of the virtual machine, and improves the performance of the virtual machine.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a schematic flowchart of a virtual environment data real-time replication method according to embodiment 1 of the present invention;
fig. 2 is a schematic diagram illustrating transmission between a virtual machine and an external data receiving device in a virtual environment data real-time replication method according to embodiment 1 of the present invention;
fig. 3 is a schematic diagram of a virtual environment data real-time replication system according to embodiment 2 of the present invention.
Detailed Description
The technical solutions of the present invention will be described clearly and completely with reference to the accompanying drawings, and it should be understood that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the description of the present invention, it should be noted that the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In the description of the present invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "virtual environment", "virtual disk", "file system backup mechanism", "data merge mechanism", "temporary storage policy", "bitmap copy mechanism", "first bitmap", "first disk space", "data receiving device", "first data", and "first physical space" should be understood in a broad sense. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
It is noted that in the description of the present invention
HA (High Availability) is highly available, and it is to reduce down time caused by daily maintenance operation (planning) and sudden system breakdown (non-planning) as much as possible by means of dual-computer hot standby and the like, so as to improve Availability of the system and application.
Transparent transmission (pass-through) refers to that the communication is only responsible for transmitting the transmitted content from the source address to the destination address regardless of the transmitted service content, and no change is made to the content of the service data.
The dirty page is a concept in the linux kernel, because the read-write speed of the hard disk is far from the speed of the memory, the system puts data with more frequent read-write into the memory in advance to improve the read-write speed, which is called as a cache, the linux takes the page as a unit of the cache, when the process modifies the data in the cache, the page is marked as the dirty page by the kernel, and the kernel writes the data of the dirty page into the disk at a proper time so as to keep the data in the cache and the data in the disk consistent.
The SAS card is a hard disk expansion card.
A VM (Virtual Machine) is a Virtual Machine, running within a virtualization hypervisor, with its own operating system.
CDP (Continuous Data Protection) is a Continuous Data Protection, which transmits and stores Data of a running computer system to a Data Protection device in real time or at short intervals, and can restore the Data to a time point when a problem occurs or at a previous time point when a protected system fails.
IP (Internet Protocol) is an Internet Protocol, which is a network layer Protocol in the TCP/IP hierarchy.
Tight coupling means that the modules or systems are in close relationship and have mutual calling.
An increment, also known as a change, refers to the amount of change in a value over a period of time.
The DB (database) system is a database system.
The I/O Input/Output (Input/Output) is divided into two parts, namely an IO device and an IO interface.
In the description of the present invention, it should be noted that the first virtual machine is a primary virtual machine, the second virtual machine is a standby virtual machine, the virtual disk of the primary virtual machine is a first virtual disk, and the virtual disk copy is a second virtual disk.
Example 1
An embodiment of the present invention provides a method for real-time replication of data in a virtualization environment, which is applied to a server carrying a virtualization environment, and please refer to fig. 1, where the method includes the following steps:
s100, setting an interface of a virtual disk in a virtualization environment, wherein the interface of the virtual disk comprises: SAS cards, RAID cards, and the like, which are exemplified as SAS cards in this embodiment;
a virtualized environment refers in this embodiment to a virtual machine,
it should be noted that, if other types of virtual disk interfaces appear in the subsequent technology, other types of virtual disk interfaces may also be used to implement the real-time copy method of data in the virtualization environment of the present invention.
The virtual SAS card records the data to be copied as first data, copies the first data through the virtual SAS card,
because the file system in the current operating system has a memory caching mechanism, most applications access the file system in a caching mode, but the DB system actually accesses the data file in a transparent transmission mode because a reliable copy needs to be kept on a disk;
the method for keeping a reliable copy on a disk means that the database file is generally written to the disk in a mode of bypassing an operating system disk cache and preventing the data file from being unreliable due to accidental power failure;
the purpose of copying by using the virtual SAS card can keep consistent with the data of the production environment, and the virtual SAS is equivalent to a disk, so that the application of a DB system is protected more obviously, and the influence on the virtual machine is smaller than the influence on the copying mode of the use agent in the virtual machine;
the virtual SAS cards of different virtual machines provide management functions through a virtualization platform or external data receiving equipment and conduct unified management, and the management content comprises IP address distribution, virtual machine and virtual SAS card matching and the like.
S200, data replication is carried out, wherein the data replication comprises data replication operation between virtual machines and data replication operation between the virtual machines and external data receiving equipment;
s201, performing data copying operation between virtual machines;
the method specifically comprises the following steps: setting a file system backup mechanism for an application using a file system memory cache mechanism, wherein the file system backup mechanism comprises the following steps: setting a backup time point, and obtaining reliable backup of the backup time point by the virtual machine in a flush dirty page mode at the backup time point;
the method comprises the following specific steps: sending an instruction to a server carrying a virtual environment, operating a certain virtual machine according to the instruction by the server carrying the virtual environment, wherein the virtual machine is a main virtual machine, and at the moment, the server carrying the virtual environment can inform the virtual machine of a cache page related to hard disk modification in a memory of the virtual machine, the cache page is a dirty page, namely a dirty page, and is put into a virtual disk file, performing snapshot operation on the virtual disk file at the backup time point, independently writing write IO operation after the backup time point into a change file, and performing data copy operation between the virtual machines on the file which is subjected to the snapshot operation;
the data copying operation between the virtual machines comprises the following steps: and transmitting the virtual disk data in the primary virtual machine after the snapshot operation to a virtual disk copy of another standby virtual machine to obtain a virtual disk copy which is the same as the virtual disk file data.
It should be noted that the copy operation between the virtual machines may be that both the two virtual machines are in one server, and the copy operation is performed through the IP address between the two virtual machines, or that the two virtual machines are on two different servers, and the copy operation is performed through the IP address between the two virtual machines and the network cards of the two servers;
and the copy operation between the virtual machines may be performed in any way, which is not limited in this embodiment.
At the moment, the main virtual machine is running, the standby virtual machine is in a shutdown state when being normal, and the standby virtual machine uses a mirror image of the main virtual machine; starting the standby virtual machine when the main virtual machine is down, and continuously executing the service processed by the main virtual machine through the standby virtual machine;
s202, setting a data merging mechanism, a temporary storage strategy and a bitmap copying mechanism, performing data copying operation between a virtual machine and external data receiving equipment through the data merging mechanism, the bitmap copying mechanism and a virtual SAS card, and sending operation data of the virtual machine on a virtual disk to the data receiving equipment;
referring to fig. 2, the specific steps include: the operation data comprises read IO operation data and write IO operation data; the virtual SAS card performs read IO and write IO operations on the virtual disk, only copies the write IO operations during data copying, and sets the IP address of the virtual SAS card; the virtual SAS card is provided with an independent IP address, and the IP address can be used as a tightly coupled data agent; this IP address can also communicate data to an external standby machine connected to the virtual machine or a dedicated program that decouples the functionality of the CDP server connected to the virtual machine onto the server hosting the virtual machine with an external data receiving device;
the virtual SAS card copies write IO operation data of the virtual disk through the bitmap copying mechanism and the data merging mechanism;
the virtual SAS card communicates with external data receiving equipment through a network card and an IP network on the server according to the IP address of the virtual SAS card, and sends data of write IO operation to the external data receiving equipment.
Further, in order to improve the efficiency of network transmission and reduce the influence on the production environment, a data merging mechanism is provided when communicating with an external data receiving device, and the data merging mechanism is as follows: merging data of write IO operation on a plurality of virtual disks, marking the merged data as first data, compressing the first data, and packaging the first data into one or a group of IP packets to be transmitted to external data receiving equipment;
in order to improve the reliability of CDP, a temporary storage strategy is set, and a physical space of a part of disks is set in a server where a virtual machine is located and is recorded as a first physical space; when data transmission between the virtual machine and the external data receiving equipment is interrupted, including unstable or disconnected IP network or failure of the external data receiving equipment, and the like, so that the data of write IO operation cannot be copied to the external data receiving equipment, the first physical space of the local disk on the server is used, and meanwhile, whether the data of the virtual disk changes is recorded by using a bitmap, so that unnecessary data transmission is avoided, and the data of the write IO operation in the virtual SAS card is temporarily stored to the first physical space; when data transmission between the virtual machine and the external data receiving equipment is recovered, including IP network stabilization or external data receiving equipment failure recovery and the like, the changed data recorded through the bitmap is written into the first physical space, the data in the first physical space is copied to the external data receiving equipment, and the probability of communication failure with the external data receiving equipment can be reduced through the temporary storage strategy.
The bitmap replication mechanism is: setting a plurality of bitmaps (bitmaps) for the virtual disk file, dividing the disk space in the virtual disk into a plurality of disk spaces, and binding the disk spaces with the bitmaps;
each bit of the bitmap corresponds to a disk space of a certain block (such as 1M or 5M/block), whether the disk space is changed or not is recorded, the unchanged bitmap is directly copied from an original virtual hard disk to external data receiving equipment, the change only occurs when write IO operation is carried out, the changed disk space is recorded as a first disk space when the change is carried out, the bitmap corresponding to the first disk space is recorded as a first bitmap, the first bitmap is marked, when the write IO operation data is copied, data in the first disk space corresponding to the first bitmap is copied, and the data are combined and sent to external data receiving equipment after the write IO operation data are written in real time, so that the whole disk is prevented from being copied firstly, then an increment is copied, if the whole disk is copied in the increment, the increment part needs to be copied for multiple times, certain disk blocks can be changed repeatedly in the copying process, and if the bitmap is used, the time and the resources of the disk are saved.
When the copying is completed, the bitmap can be deleted, and the resources are released.
It should be noted that, in the present invention, the copied data may be all files of the selected virtual machine, or may be data of a designated file of the virtual machine, that is, the real-time copying method of data in a virtualization environment provided in the embodiment of the present invention can provide data for copying different levels, may be complete data of a file of the virtual machine, or may be data of a file level of the virtual machine, and a user may select the data according to a requirement, and the real-time copying method of data in a virtual environment of the present invention is also applicable to the HA.
Example 2
An embodiment of the present invention further provides a virtual environment data real-time replication system, please refer to fig. 3, including: the system comprises a creation module, a file system backup module, a bitmap copying module and a data copying module;
the creating module is used for setting a virtual disk interface in the virtual environment, and the virtual disk interface comprises: SAS cards, RAID cards, etc.;
the file system backup module is used for setting a backup time point and performing backup in a dirty page mode at the backup time point;
specifically, the method comprises the following steps: sending an instruction to a server carrying a virtual environment, wherein the server carrying the virtual environment sends the instruction to a main virtual machine according to the instruction, and at the moment, the server carrying the virtual environment informs the main virtual machine to place dirty pages, namely dirty pages, in a memory of the virtual machine into a virtual disk file, the virtual disk file of the main virtual machine is subjected to snapshot operation at the backup time point, write IO (input/output) operation occurring after the backup time point is independently written into a change file, and the file subjected to the snapshot operation is subjected to data copying operation among the virtual machines;
the bitmap copying module is used for setting a plurality of bitmaps, dividing the space in the virtual disk into a plurality of disk spaces, binding the disk spaces and the bitmaps, marking operation data through the bitmaps and copying the operation data according to a marking result;
specifically, the method comprises the following steps: when the virtual disk is subjected to write operation data, recording a disk space for performing the write operation data as a first disk space, matching a first bitmap through the first disk space, and marking the first bitmap;
when data copying is carried out, copying the first disk space corresponding to the first bitmap;
the data copying module is used for copying data between virtual environments through the file system backup module, copying data receiving equipment through the bitmap copying module and the virtual disk interface, and sending operation data of the virtual disk to the data receiving equipment.
The system also comprises a data merging module and a temporary storage module;
the data merging module is used for merging write IO operation data, recording the merged write operation data as first data, compressing the first data, packaging the first data into an IP packet and transmitting the IP packet to the data receiving equipment;
the temporary storage module is used for setting a first physical space of a disk in the server;
the temporary storage module is used for storing the data of the write operation data into the first physical space when the data transmission between the virtual machine and the external data receiving equipment is interrupted, including when the network is unstable or disconnected, and simultaneously recording whether the data of the virtual disk changes or not by using a bitmap;
and the temporary storage module is used for writing the changed virtual disk data recorded by the bitmap into the first physical space and transmitting the data in the first physical space to the data receiving equipment when the data transmission between the virtual machine and the external data receiving equipment is recovered, including when the network is stable.
Example 3
The present invention also provides a computer readable storage medium, which stores a computer program for executing the above real-time virtual environment data copying method when executed by a processor, and implements the steps of the above real-time virtual environment data copying method.
Finally, it should be noted that, as one of ordinary skill in the art can appreciate, all or part of the processes of the methods of the above embodiments may be implemented by a computer program to instruct related hardware, and the program of the method of monitoring software may be stored in a computer readable storage medium, and when executed, may include the processes of the embodiments of the methods as described above. The storage medium of the program may be a magnetic disk, an optical disk, a Read Only Memory (ROM), a Random Access Memory (RAM), or the like. The embodiments of the computer program may achieve the same or similar effects as any of the above-described method embodiments.
Furthermore, the methods disclosed according to embodiments of the present invention may also be implemented as a computer program executed by a processor, which may be stored in a computer-readable storage medium. Which when executed by a processor performs the above-described functions defined in the methods disclosed in embodiments of the invention.
Further, the above method steps and system elements may also be implemented using a controller and a computer readable storage medium for storing a computer program for causing the controller to implement the functions of the above steps or elements.
Further, it should be appreciated that the computer-readable storage media (e.g., memory) herein can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory.
Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as software or hardware depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosed embodiments of the present invention.
The numbers of the embodiments disclosed in the above embodiments of the present invention are merely for description, and do not represent the advantages or disadvantages of the embodiments.
The above description is only an embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes performed by the present specification and drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (4)
1. A real-time virtual environment data copying method is applied to a server carrying a virtual environment, and is characterized by comprising the following steps: setting a virtual disk interface in the virtual environment;
setting a file system backup mechanism, and performing data copying operation between virtual environments through the file system backup mechanism; the file system backup mechanism is as follows: setting a backup time point, wherein the virtual environment is backed up in a dirty page mode at the backup time point;
setting a data merging mechanism, a temporary storage strategy and a bitmap copying mechanism, and performing data copying operation between a virtual environment and data receiving equipment through the data merging mechanism, the bitmap copying mechanism and the virtual disk interface; the bitmap copying mechanism is as follows: setting a plurality of bitmaps, marking operation data through the bitmaps, and copying the operation data according to a marking result;
the bitmap copying mechanism is further as follows: dividing the space in the virtual disk into a plurality of disk spaces, and binding the disk spaces with the bitmap; the operation data comprises read operation data of the virtual disk and write operation data of the virtual disk; when the write operation data is carried out, recording a disk space for carrying out the write operation data as a first disk space, matching a first bitmap through the first disk space, and marking the first bitmap; when the write operation data is copied, copying the data in the first disk space corresponding to the first bitmap;
the step of performing a data copy operation between a virtual environment and a data receiving device via the data merging mechanism, the bitmap copying mechanism, and the virtual disk interface further includes: setting an IP address of a virtual disk interface; the virtual disk interface copies the write operation data through the bitmap copying mechanism and the data merging mechanism; establishing communication with data receiving equipment through the IP address, a network card on the server and a network, and transmitting the write operation data to the data receiving equipment;
the data merging mechanism is as follows: merging the write operation data, and recording the merged write operation data as first data; compressing the first data, packaging the first data into an IP packet and transmitting the IP packet to the data receiving equipment;
the temporary storage policy is: setting a first physical space of a disk in the server; when the data transmission between the virtual environment and the data receiving equipment is interrupted, storing the data of the write operation data into the first physical space, and simultaneously recording whether the data in the virtual disk changes or not by using a bitmap; when the data transmission between the virtual environment and the data receiving equipment is recovered, writing the changed data of the virtual disk recorded by the bitmap into the first physical space, and transmitting the data in the first physical space to the data receiving equipment;
the file system backup mechanism further comprises: sending an instruction to the server, wherein the server operates the virtual environment according to the instruction, and records the virtual environment which is operated as a first virtual environment; the dirty pages in the memory of the first virtual environment are placed into a first virtual disk of the first virtual environment, snapshot operation is carried out on the first virtual disk at the backup time point, and the write operation data appearing after the backup time is written into a change file; data copy operations between virtual environments are performed.
2. The method of claim 1, wherein the method comprises: the step of performing a data copy operation between virtual environments further comprises: sending the data of the first virtual disk in the first virtual environment after the snapshot operation to a second virtual disk in a second virtual environment in the server to obtain the second virtual disk with the same data as the first virtual disk;
and the second virtual machine uses the mirror image of the first virtual machine, starts the second virtual machine when the first virtual machine is down, and uses the second virtual machine to execute the service of the first virtual machine.
3. A virtual environment data real-time replication system, comprising: the system comprises a creating module, a file system backup module, a bitmap copying module, a data merging module and a temporary storage module;
the creating module is used for setting a virtual disk interface in a virtual environment;
the file system backup module is used for setting a backup time point, and performing backup on the virtual environment in a dirty page mode at the backup time point; the method specifically comprises the following steps: sending an instruction to a server carrying a virtual environment, wherein the server carrying the virtual environment informs a main virtual machine to put dirty pages in a memory of the virtual machine into a virtual disk file, carrying out snapshot operation on the virtual disk file of the main virtual machine at a backup time point, independently writing write operation occurring after the backup time point into a changed file, and carrying out data copying operation between the virtual machines on the file carrying out the snapshot operation;
the bitmap copying module is used for setting a plurality of bitmaps, dividing the space in the virtual disk into a plurality of disk spaces, binding the disk spaces and the bitmaps, marking operation data through the bitmaps and copying the operation data according to a marking result; the method specifically comprises the following steps: when writing operation data to the virtual disk, recording the disk space of the writing operation data as a first disk space, matching a first bitmap through the first disk space, and marking the first bitmap; when data copying is carried out, copying the first disk space corresponding to the first bitmap;
the data copying module is used for performing data copying operation between virtual environments through the file system backup module;
the data copying module is used for copying data between the virtual environment and the data receiving equipment;
the data merging module is used for merging the write operation data, recording the merged write operation data as first data, compressing the first data, packaging the first data into an IP packet and transmitting the IP packet to the data receiving equipment;
the temporary storage module is used for setting a first physical space of a disk in the server;
the temporary storage module is used for storing the data of the write operation data into the first physical space when the data transmission between the virtual environment and the data receiving equipment is interrupted, and simultaneously recording whether the data of the virtual disk changes or not by using a bitmap;
the temporary storage module is configured to, when data transmission between the virtual environment and the data receiving device is resumed, write the changed virtual disk data recorded by the bitmap into the first physical space, and transmit data in the first physical space to the data receiving device.
4. A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, carries out the steps of a method for real-time replication of virtual environment data according to claim 1 or 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110322237.1A CN113127151B (en) | 2021-03-25 | 2021-03-25 | Real-time copying method, system and medium for data in virtualization environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110322237.1A CN113127151B (en) | 2021-03-25 | 2021-03-25 | Real-time copying method, system and medium for data in virtualization environment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113127151A CN113127151A (en) | 2021-07-16 |
| CN113127151B true CN113127151B (en) | 2023-03-21 |
Family
ID=76774111
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110322237.1A Active CN113127151B (en) | 2021-03-25 | 2021-03-25 | Real-time copying method, system and medium for data in virtualization environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113127151B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104298559A (en) * | 2014-09-30 | 2015-01-21 | 深圳市深信服电子科技有限公司 | Method and device for virtualizing physical host system |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4292882B2 (en) * | 2003-03-18 | 2009-07-08 | 株式会社日立製作所 | Plural snapshot maintaining method, server apparatus and storage apparatus |
| JP4615595B2 (en) * | 2008-12-22 | 2011-01-19 | 富士通株式会社 | Storage switch, storage system, and data copy method |
| US9842032B2 (en) * | 2013-08-27 | 2017-12-12 | Red Hat, Inc. | Memory first live snapshot |
| US9898320B2 (en) * | 2014-06-28 | 2018-02-20 | Vmware, Inc. | Using a delta query to seed live migration |
| TWI584131B (en) * | 2015-12-14 | 2017-05-21 | 財團法人工業技術研究院 | Server backup method and backup system thereof |
| CN106802840A (en) * | 2017-01-19 | 2017-06-06 | 郑州云海信息技术有限公司 | A kind of virtual machine backup, restoration methods and device |
-
2021
- 2021-03-25 CN CN202110322237.1A patent/CN113127151B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104298559A (en) * | 2014-09-30 | 2015-01-21 | 深圳市深信服电子科技有限公司 | Method and device for virtualizing physical host system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113127151A (en) | 2021-07-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9563684B1 (en) | Replication cookie | |
| US9336230B1 (en) | File replication | |
| US8600945B1 (en) | Continuous data replication | |
| US9740573B1 (en) | Dynamic LUN resizing in a replication environment | |
| US9875042B1 (en) | Asynchronous replication | |
| US8725692B1 (en) | Replication of xcopy command | |
| US7127578B2 (en) | Storage device and information management system | |
| EP2593867B1 (en) | Virtual machine aware replication method and system | |
| US8738813B1 (en) | Method and apparatus for round trip synchronous replication using SCSI reads | |
| US8850143B1 (en) | Point in time access in a replication environment with LUN resizing | |
| US9471579B1 (en) | Replicating selected snapshots from one storage array to another, with minimal data transmission | |
| US8977593B1 (en) | Virtualized CG | |
| US8726066B1 (en) | Journal based replication with enhance failover | |
| US8261035B1 (en) | System and method for online data migration | |
| US8335771B1 (en) | Storage array snapshots for logged access replication in a continuous data protection system | |
| US6950915B2 (en) | Data storage subsystem | |
| JP4809040B2 (en) | Storage apparatus and snapshot restore method | |
| US7103713B2 (en) | Storage system, device and method using copy-on-write for synchronous remote copy | |
| US7293068B2 (en) | Method and system for storing duplicate data | |
| CN106357703B (en) | Cluster switching method and device | |
| US20040225914A1 (en) | Method, apparatus and program storage device for allowing continuous availability of data during volume set failures in a mirrored environment | |
| CN108182128A (en) | Based on XEN without Agent virtual back-up restoring method | |
| CN113127151B (en) | Real-time copying method, system and medium for data in virtualization environment | |
| US7761737B2 (en) | Apparatus, method, and computer product for executing program | |
| US11080242B1 (en) | Multi copy journal consolidation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |