WO2020042852A1 - Method for copying data, and master device, and slave device - Google Patents

Method for copying data, and master device, and slave device Download PDF

Info

Publication number
WO2020042852A1
WO2020042852A1 PCT/CN2019/098307 CN2019098307W WO2020042852A1 WO 2020042852 A1 WO2020042852 A1 WO 2020042852A1 CN 2019098307 W CN2019098307 W CN 2019098307W WO 2020042852 A1 WO2020042852 A1 WO 2020042852A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
slave device
version
request message
master device
Prior art date
Application number
PCT/CN2019/098307
Other languages
French (fr)
Chinese (zh)
Inventor
孙嘉岑
钱海峰
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2020042852A1 publication Critical patent/WO2020042852A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/065—Replication mechanisms
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes

Definitions

  • the present application relates to the field of storage, and more particularly, to a method for copying data, a master device, and a slave device.
  • the present application provides a method and device for copying data, which can reduce the size of metadata and improve the performance of copying data.
  • a method for copying data including: a master device sends a first request message to a slave device, the first request message is used to instruct a slave device to obtain second data, and the second device
  • the data covers the first data, the master device and the slave device are respectively located in different data centers, the first data and the second data are two different versions of data of the same name, and the data of the second data A version later than the version of the first data;
  • the first request message can be used to instruct the slave device to obtain the second data and cover the first data Data
  • the master device receives a first response message sent by the slave device, the first response message is used to indicate that the slave device successfully copied the second data;
  • the master device When the latest version of the data of the same name stored in the slave device is not the first data but the third data, the master device receives a second request message sent by the slave device, and the second request The message is used to query an overlay relationship between the second data and the third data in the master device, where the third data is data of a version of the data of the same name;
  • the master device sends second instruction information to the slave device, where the second instruction information is used to indicate an overlay relationship between the second data and the third data in the master device, and the master device Receiving a first response message sent by the slave device, where the first response message is used to indicate that the slave device successfully copied the second data.
  • the above copy data can be understood as a copy object, a copy file, or a copy block.
  • asynchronous second copy of the second data is performed through the first request message sent from the master device to the slave device, and the first saved message in the slave device is overwritten according to the first request message.
  • the first data is the old version of the data with the same name
  • the second data is the new version of the data with the same name.
  • the metadata of the second data does not need to add a metadata clock to record the operation history, reduce the size of the metadata, and improve the performance of copying data.
  • the slave cannot determine the second data and the first data. Coverage relationship of three data.
  • a back-to-source check mechanism is added, that is, the master device receives a second request message sent by the slave device for querying the coverage relationship between the third data and the second data, and the master device In the coverage relationship in, the second instruction information is sent to the slave device, and the coverage relationship between the third data and the second data is re-indicated. Can guarantee the final consistency between the master device and the slave device when data is copied.
  • the second indication information may specifically be status information of the third data and the second data, or
  • the second indication information may specifically be the time when the third data and the second data are off.
  • the clock trusted means that the system where the master device and the slave device are located is provided with an atomic clock, and the clock information on the master device side is accepted from the device side. It can provide flexibility and selectivity for possible indication forms of the second indication information.
  • the state of the second data is the deleted (overwritten) state
  • the third data is the data that covers the second data
  • the state of the third data is Non-deleted (not overwritten) state
  • the first request message includes first indication information, and the first indication information is used to indicate that a version of the second data is later than the first The version of the data.
  • the slave device by carrying indication information that the version of the second data is later than the version of the first data in the first request message, the slave device receives the first When a request message is received, the coverage relationship between the second data and the first data can be determined according to the first instruction information carried in the request message. Improve the accuracy of copied data.
  • the first request message further includes second data, or includes a first identifier of the second data.
  • the slave device When the first request message includes the second data, the slave device directly receives the second data and saves the second data. That is, an implementation of obtaining the second data from the device;
  • the slave device When the first request message includes the first identifier of the second data, the slave device obtains the second data from the master device according to the first identifier of the second data, and copies the second data. That is, another implementation of obtaining the second data from the device.
  • the first identification of the second data may be a version number of the second data, or other identification information capable of indicating the second data.
  • the first request message may not include the foregoing first indication information, and includes the following two possible situations:
  • the above-mentioned master device and slave device include a global scheduler between different data centers.
  • the scheduler can allocate a globally unique second identifier for each version of data uploaded with the same name data, and the The second identifier is an increasing sequence, that is, the second identifier of the new version must be greater than the second identifier of the old version.
  • the first request message may include only the second identifier of the second data, or the second identifier and the second data of the second data.
  • the slave device obtains the second data from the master device according to the second identifier of the second data, and copies the second data. That is, an implementation of obtaining the second data from the device.
  • the slave device directly receives the second data and saves the second data. That is, the second implementation of obtaining the second data from the slave device:
  • the above-mentioned master device and the slave device include a global scheduler between different data centers, and the scheduler can upload data for each version of the same name data. Data, assign a globally unique clock (there is an atomic clock in the system), and this clock can accurately represent the time when each version of the data of the same name is in the master device, that is, the new version of the clock must be later than the old version Clock.
  • the first request message may include only the clock information of the second data, or the clock information of the second data and the second data.
  • the slave device when only the clock information of the second data is included in the first request message, the slave device obtains the second data from the master device according to the clock information of the second data, and copies the second data. That is, an implementation of obtaining the second data from the device.
  • the slave device when the clock information and the second data of the second data are included in the first request message, the slave device directly receives the second data and saves the second data. That is, another implementation of obtaining the second data from the device.
  • the first indication information is carried in a header field of the first request message.
  • the first indication information may be carried in a header field of the first request message.
  • the slave device receives the first request message, it can quickly obtain the version of the second data later than the version of the first data, and then learn the coverage relationship between the first data and the second data.
  • the first indication information may be carried in another position of the first request message.
  • the first indication information is a version number of the first data.
  • the first indication information may be a version number of the first data. Since the version numbers of data of different versions of the same name data can uniquely determine the data corresponding to the version number, the first indication information is the version number of the first data, which can improve the accuracy of determining the first data from the device.
  • the method before the master device sends a first request message to a slave device, the method further includes: the master device determines the The state of the second data, the state of the second data is an uncovered state.
  • the master device before the master device sends a first request message instruction to the second data to acquire the second data from the device, the master device determines that the state of the second data is an uncovered state. Two data send the first request message. It can avoid redundancy of duplicated information and save signaling overhead.
  • a method for copying data including: receiving a first request message sent by a master device from a slave device, the first request message being used to instruct the slave device to obtain second data, and The two data cover the first data, the master device and the slave device are respectively located in different data centers, the first data and the second data are two different versions of data of the same name, and the second data The version of is later than the version of the first data;
  • the slave device When the latest version of the data of the same name stored in the slave device is the first data, the slave device obtains the second data according to the first request message, and overwrites the second data locally The saved first data; the slave device sends a first response message to the master device, the first response message is used to indicate that the slave device successfully copied the second data; or,
  • the slave device When the latest version of the data of the same name stored in the slave device is not the first data but the third data, the slave device sends a second request message to the master device, and the second request message For querying the coverage relationship between the second data and the third data, wherein the third data is data of one version of the data of the same name; and the slave device receives a second instruction sent by the master device Information, and the second indication information is used to indicate a coverage relationship between the second data and the third data.
  • a slave device receives a first request message sent by a master device, performs asynchronous copying of the second data, and overwrites the first data according to the first request message.
  • the first data is the old version of the data with the same name
  • the second data is the new version of the data with the same name.
  • the metadata of the second data does not need to add a metadata clock to record the operation history, reduce the size of the metadata, and improve the performance of copying data.
  • the slave device When the first instruction information indicates that the second data covers the first data, and the slave device does not store the first data, and the slave device stores the third data, the slave device cannot determine the overlay relationship between the second data and the third data.
  • a back-to-source check mechanism is added, that is, the slave device sends a second request message for querying the coverage relationship between the third data and the second data to the master device, and the master device stores the second request message in itself according to the third data and the second data. Sending the second indication information to the slave device to re-indicate the coverage relationship between the third data and the second data. Can guarantee the final consistency between the master device and the slave device when data is copied.
  • the second indication information may specifically be status information of the third data and the second data, or a next disk time of the third data and the second data. It can provide flexibility and selectivity for possible indication forms of the second indication information.
  • the state of the second data is a deleted (overwritten) state
  • the third data is the data that covers the second data
  • the third data The state of is the non-deleted (not overwritten) state
  • the first request message includes first indication information, and the first indication information is used to indicate that a version of the second data is later than the first data The version of the data.
  • the slave device by carrying indication information that the version of the second data is later than the version of the first data in the first request message, the slave device receives the first When a request message is received, the coverage relationship between the second data and the first data can be determined according to the first instruction information carried in the request message. Improve the accuracy of copied data.
  • the first indication information is carried in a header field of the first request message.
  • the first indication information may be carried in a header field of the first request message.
  • the slave device receives the first request message, it can quickly obtain the coverage relationship between the first data and the second data.
  • the first indication information may be carried in another position of the first request message.
  • the first indication information is a version number of the first data.
  • the first indication information may be a version number of the first data. Because the version numbers of data of different versions of the same name data can uniquely determine the data corresponding to the version number, the first indication information is the version number of the first data, which can improve the accuracy of determining the first data from the device.
  • a master device includes each unit for performing the foregoing first aspect and the method in any possible implementation manner of the first aspect.
  • a slave device includes each unit for performing the foregoing second aspect and the method in any possible implementation manner of the second aspect.
  • a host device including at least one processor and at least one memory.
  • the at least one memory is configured to store a computer program
  • the at least one processor is configured to call and run the computer program from the at least one memory, so that the main device executes the foregoing first aspect and the method in any possible implementation manner of the first aspect.
  • the main device also includes a hard disk, which is used to store the data of the same name.
  • a slave device including at least one processor and at least one memory.
  • the at least one memory is configured to store a computer program
  • the at least one processor is configured to call and run the computer program from the memory, so that the slave device executes the foregoing second aspect and the method in any possible implementation manner of the second aspect
  • the The slave device also includes a hard disk for storing the data of the same name.
  • a system including a master device in the fifth aspect and a slave device in the sixth aspect.
  • a computer program product includes computer program code that, when the computer program code runs on a computer, causes the computer to execute the methods in the first and second aspects described above.
  • the above computer program code may be stored in whole or in part on a first storage medium, where the first storage medium may be packaged with the processor, or may be packaged separately with the processor. This embodiment of the present application does not deal with this. Specific limitations.
  • a computer-readable medium stores program code, and when the computer program code runs on a computer, the computer causes the computer to execute the methods in the first and second aspects. .
  • the method for copying data, the master device, and the slave device proposed in this application do not need to record the operation history of the data in the metadata of the data to be copied, which can reduce the size of the metadata and improve the performance of copying the data.
  • FIG. 1 is a schematic diagram of a scenario to which a method for copying data provided by an embodiment of the present application is applied.
  • Figure 2 is a schematic diagram of a method based on metaclock asynchronous replication.
  • Figure 3 is a schematic diagram of another asynchronous replication method based on metaclock.
  • FIG. 4 is a schematic diagram of a method for copying data proposed by the present application.
  • FIG. 5 is a schematic diagram of a specific embodiment of a method for copying data provided by the present application.
  • FIG. 6 is a schematic diagram of a main device according to an embodiment of the present application.
  • FIG. 7 is a schematic block diagram of a master device according to another embodiment of the present application.
  • FIG. 8 is a schematic diagram of a slave device according to an embodiment of the present application.
  • FIG. 9 is a schematic block diagram of a slave device according to another embodiment of the present application.
  • FIG. 1 is a schematic diagram of a scenario to which a method for copying data provided by an embodiment of the present application is applied. Includes 110 and 120 parts.
  • the 110 is a data center, and specifically includes a data center 111 and a data center 112 as shown in FIG. 1.
  • the data center 111 and the data center 112 are two different data centers.
  • the data center 111 and the data center 112 may be located in two different areas.
  • the data center 111 and the data center 112 may be different data centers in the same area.
  • a device participating in the data copy process in the data center 111 may be referred to as a master device, and a device participating in the data copy process in the data center 112 may be referred to as a slave device; or,
  • the devices participating in the data replication process in the data center 111 may also be referred to as slave devices, and the devices participating in the data replication process in the data center 112 may be referred to as master devices. This application is not limited to this.
  • slave devices is not limited in the embodiments of the present application, and data replication may be performed between multiple slave devices and one master device.
  • FIG. 1 also includes multiple data centers, and each data center includes a slave device, and data is replicated between the master device and the slave device.
  • a master device can also be called a master node, a master cluster, a master region (region), or a master end;
  • a slave device can also be called a slave node, a slave cluster, a slave region, or a slave end, and so on.
  • 120 is a copy channel, and is used to support transmission of data to be copied between the master device and the slave device.
  • the device that migrates data from the data center where the data is migrated can be referred to as the above-mentioned master device.
  • the device that migrates data is called the above-mentioned slave device.
  • the master device and the slave device shown in FIG. 1 are the largest serviceable units of centralized metadata management, and for example, may be a network device or a terminal device including a processor and a memory.
  • Asynchronous replication of data is performed without affecting the latency of the local data.
  • the data disk specifically refers to: a way to implement data persistence, for example, storing data from a volatile memory to a non-volatile hard disk.
  • the replication of data is performed in an asynchronous form, and the replication of data by the master device and the execution of the data are performed by two threads without affecting each other.
  • the master device first receives the first version of the data of the same name uploaded by the user, sends a data copy request to the slave device, and requests the slave device to copy the first version of the data of the same name.
  • the master device On the master device, the first disk version of the data of the same name is downloaded to the disk, and the data copy of the first version is independent.
  • final consistency refers to: two pieces of data with the same name stored in the master device and the slave device, and multiple overwrites between the master device and the slave device for data of different versions of the same name data, and Asynchronous replication between the master device and the slave device can ensure that after a certain period of time, the latest versions of the data of the same name in the master device and the slave device are consistent.
  • Data with the same name refers to data with the same name in the database.
  • the new data when adding new data to the database, if there is data in the database with the same name as the data to be added, the new data overwrites the old data.
  • the coverage of the above data refers to the coverage that occurs between multiple versions of data of the same name, because in a non-multi-version scenario, the database can only include one version of data of the same name.
  • the master device and / or the slave device are enabled, there is no case where data of multiple versions of the same name data is overwritten. Because, when the master device and / or the slave device enable multiple versions, multiple versions of the same name data are retained.
  • the data replication involved in this application refers to the asynchronous replication of data of the same name between different devices across data centers without multi-version scenarios enabled.
  • the master device and the slave device may use meta clocks to store the coverage history of data of different versions with the same name.
  • each meta clock saves the creation time of the historical data (including the logical clock and the machine clock), and the operating context at that time. Comparing the two meta clock lists of the two versions of the data with the same name, we can get the timing of the next disk of the two versions of the data.
  • the above-mentioned master device and slave device can copy data because the two master devices and slave devices across the data center have a protocol implementation or control mechanism capable of reliably transmitting data, and shielding the data transmission itself may Consistency interference factor.
  • Concurrent transaction An optimistic lock implementation of data concurrent transactions.
  • Optimistic Lock is a method of concurrent control. Assume that concurrent transactions of multiple users do not affect each other during processing, and each transaction can process the part of data that it affects without generating a lock. Before committing the data update, each transaction first checks whether any other transaction has modified the data after the transaction reads the data. If other transactions are updated, the committing transaction will be rolled back.
  • Data version refers to adding a version identifier to the data.
  • a version solution based on a database table it is generally implemented by adding a "version" field to the database table.
  • the version data of the submitted data is compared with the current version information of the corresponding record of the database table. If the version number of the submitted data is greater than the current version number of the database table, it is updated, otherwise it is considered to be outdated data.
  • WAL Write-ahead logging
  • the log file usually includes redo and undo information.
  • the program may need to know whether the operation performed at that time was successful, partially successful, or failed. If WAL is used, the program can check the log file and compare the content of the operation planned to be performed with the actual operation during a sudden power failure. Based on this comparison, the program can decide whether to undo the operation or continue to complete the operation, or leave it as it is.
  • the meta clock includes information such as data creation time (createTime) and request information (requestInfo) of different versions of data of the same name.
  • the metadata of different versions of the same name data uses the meta clock field to store the N operation history of this data, and the meta clock in the metadata corresponding to the data of different versions of the same name data is compared to determine the difference of the data of the same name.
  • the version of the data has successively covered the relationship.
  • the following describes the method for asynchronous replication based on metaclock by taking a copy request of two versions of data of the same name data received from the device and judging the overwrite relationship between the two versions of data as an example.
  • comparing the meta clock in the metadata of the two versions of data from the device is divided into two steps:
  • Step 1 The slave uses the common substring-based method to merge the meta clock in the metadata of the two versions of the data with the same name, and the non-public part of the meta clock in the metadata of the two versions of the data with the same name. Sort by time.
  • the common substrings refer to different substrings, all of which appear in their respective parent strings and appear in the same order as the parent strings.
  • the different substrings are called common substrings of different parent strings.
  • the substrings bo, bg, lg appear in both the parent string cnblogs and belongs, and the appearance order is consistent with the parent string.
  • the substrings bo, bg, lg are called common substrings.
  • the slave device judges, according to the merged meta clock, the order of the meta clock in the metadata of the two versions of the data of the same name.
  • Figure 2 is a schematic diagram of a method based on metaclock asynchronous replication. Including master and slave devices.
  • the main device uploads data of multiple versions of the same name data before and after.
  • it includes two versions of data (as shown in FIG. 2, data V9 and data V8), and data V9 covers data V8. That is, the latest version of the data of the same name in the master device is data V9.
  • V9 is the version number of the data V9.
  • the data in parentheses after V9 in FIG. 2 indicates the time stamp (1,2, 3, 7) corresponding to the metaclock included in the metadata of the data V9.
  • V8 is the version number of the data V8.
  • the data in parentheses after V8 in FIG. 2 indicates the time stamp (2, 3, 4, 5, 6) corresponding to the metaclock included in the metadata of the data V8.
  • the process of implementing asynchronous replication between the master and slave devices is mainly:
  • the master device sends a first copy data request message to the slave device for the data V9, and requests the slave device to copy the data V9; the master device sends a second copy data request message to the slave device to the data V8, and requests the slave device to copy the data V8.
  • the data V9 since the data V9 is a new version of the data of the same name, the data V9 covers the data V8 in the master device.
  • the first copy data request message sent by the master device to the slave device is sent before the second copy data request message, and the first copy data request message arrives at the slave device before the second copy data request message.
  • the first copy data request message sent by the master device to the slave device is sent before the second copy data request message, but the first copy data request message arrives later than the second copy data request message. That is, the timing at which the copy data request message sent by the master device to the slave device reaches the slave device crosses.
  • the slave device determines the data V9 and according to the metaclock in the metadata of the data V9 carried by the first copy data request message and the metaclock in the metadata of the data V8 carried in the second copy data request message. Data V8 successively covers the relationship.
  • the slave device does not compare the meta clock in the metadata of data V8 and data V9, and directly determines the coverage of data V9 and data V8 according to the sequence relationship between the first copy data request message and the second copy data request message Relationship, the latest version of the data with the same name in the slave device is data V8.
  • the slave device determines the coverage relationship between the data V8 and the data V9 by comparing the meta clock in the metadata of the data V8 and the data V9.
  • the slave uses the common substring-based merge method to merge the meta clock in the metadata of data V8 and data V9.
  • the timestamp corresponding to the meta clock in the metadata of the data V9 is: 1, 3, 2, 7;
  • the timestamp corresponding to the meta clock in the metadata of the data V8 is: 3, 2, 4, 5, 6.
  • the slave sorts the non-public part of the meta clock in the metadata of the data V8 and data V9 according to time.
  • the final result is: 1, 3, 2, 4, 5, 6, 7. Since the timestamp "7” is the timestamp corresponding to the metaclock in the metadata of the data V9, and the timestamp "7” is the latest timestamp in the metaclock result of the merged data V8 and the data V9.
  • the slave device judges that the data V9 is the latest version of the data of the same name, and overwrites the data V8.
  • the asynchronous replication method based on meta clock needs to add meta clock to the metadata of each version of the data of the same name, which increases the cost.
  • the master device needs to pre-read the metadata of each version of the data of the same name, which affects the performance of asynchronous replication.
  • Figure 3 is a schematic diagram of another asynchronous replication method based on metaclock. Including master and slave devices.
  • data V11 may be the oldest historical version, and then the master device uploads 11 data in sequence, the versions are V0-V11. It is assumed that V11 finally reaches the slave device.
  • the length of the meta clock list in the master device is limited to 11.
  • the data history version of the data with the same name is V11-V9. When the data V10 reaches the master device, the data V11 will be deleted from the meta clock list example.
  • the slave device can determine that the data V11 is the latest version of the data of the same name, and eventually the data is inconsistent.
  • This application proposes a method for copying data, which can be used to reduce the metadata size of the data and achieve the consistency of data between the master device and the slave device when used for the asynchronous data copying described above.
  • the method for copying data in the present application is described in detail below with reference to FIGS. 4 and 5. It can be applied to at least two data centers.
  • the following uses a master device and a slave device respectively in two different data centers as examples to briefly introduce the method for copying data in this application.
  • FIG. 4 is a schematic diagram of a method for copying data proposed by the present application. Including the master device, slave device and S410-S450 five steps, the five steps are described in detail below.
  • the copy data includes: object copy, file copy, block copy, and the like.
  • the data copy between the master device and the slave device is for data that needs to be copied, that is, the master device is sending data to the slave device. Before the device sends a copy data request message, it determines whether the data needs to be copied.
  • FIG. 4 includes S410, and the master device determines a state of the second data.
  • the number of times of information transmission between the slave device and the master device In order to reduce the number of data coverage scenarios between multiple versions of data with the same name, the number of times of information transmission between the slave device and the master device.
  • the master device sends a first request message to the slave device for the second data and instructs the slave device to obtain the second data, and determines the state of the second data, and sends the second data to the slave device when the state of the second data is not overwritten.
  • the state in which the second data is not overwritten may also be referred to as a non-deleted state, and the first request message may also be referred to as a copy data request message.
  • the master device determines whether to send a copy data request message for the data of a certain version according to the state of the data of the same name. Whether to send a copy data request message.
  • the copy data request message is a general summary of the foregoing first request message. It should be understood that the foregoing first request message is a copy data request message sent for the second data.
  • the master device determines whether to send a copy data request message for the data to the slave device according to the status of the data, including the following two cases:
  • Case 1 The state of the data in the master device is a deleted (overwritten) state. That is, the data has been overwritten by data newer than the data version in the master device. If the master device determines that the status of the data is deleted, the master device no longer sends a copy data request message for the data.
  • the master device receives data uploaded by the user for one version of data A of the same name (referred to as the first data), and subsequently, the master device receives data uploaded by the user for another version of data A of the same name (referred to as the first data).
  • the second data is a new version of the data, that is, the second data overwrites the first data.
  • the master device When the master device is ready to send a copy data request message for the first data to the slave device, it is determined that the first data has been overwritten and is in a deleted state. Then, the master device no longer sends a copy data request message for the first data to the slave device.
  • Case 2 The state of the data in the master device is a non-deleted (not overwritten) state, that is, the data has not been overwritten in the master device.
  • the master device determines that the state of the data is a non-deleted state, then the master device sends a copy data request message for the data.
  • the master device receives data uploaded by the user for one version of data A of the same name (referred to as the first data), and subsequently, the master device receives data uploaded by the user for another version of data A of the same name (referred to as the first data).
  • the second data is a new version of the data, that is, the second data overwrites the first data.
  • the master device When the master device is ready to send a copy data request message for the first data to the slave device, it is determined that the first data is stored in the master device and is in an undeleted state. Due to asynchronous replication, the first data has not been overwritten by the second data at this time. . Then, the master device sends a copy data request message for the first data to the slave device.
  • the moment when the master device is ready to send a copy data request message for the first data to the slave device may be the time when the master device receives another version of data uploaded by the user for the same name data A Called the second data). That is, the master device sends the data copy request message and receives the user upload data as two threads that do not interfere with each other, and can be performed simultaneously.
  • the master device when determining that the data needs to be copied, the master device sends a copy data request message for the data to the slave device.
  • the number of slave devices is not limited, and the master device may simultaneously send a copy data request message for the data to multiple slave devices.
  • the master device when the above second data is in an uncovered state, the master device requests the slave device to copy the second data as an example, and details how the master device requests the slave device to copy the second data in the embodiment of the present application.
  • FIG. 4 includes S420, and the master device sends a first request message to the slave device.
  • the slave device When the above master device sends a first request message, it instructs the slave device to obtain the second data and overwrite the second data with the first data, including the following situations:
  • Case 1 The second data mentioned above requests the master device to copy the data of the same name data for the first time, and there is no overwriting action of data of the same name and different versions of data in the master device.
  • the second data is the first version of the data with the same name. And, no other version of data in the master device is overwritten by the second data. Then, the first request message does not need to instruct the slave device to overwrite the second data with a certain data.
  • the slave device After receiving the first request message, the slave device directly copies the second data according to the first request message. Because the second data is the first data to be copied, and the second data does not cover other versions of the data, that is, there is no other version of the data of the same name to which the second data belongs in the slave device, Therefore, in the slave device, no conflict of data downloading with different versions of the same name data will occur.
  • the second data is data of the first version of the same-named data A uploaded by the user to the master device, specifically, the version number of the second data is V2;
  • the master device sends a first request message to the slave device.
  • the slave device receives the first request message, stores V2 into the metadata corresponding to the data A of the same name, and obtains the second data.
  • one possibility of obtaining the second data from the device is to directly receive the second data and save the second data; or,
  • the slave device obtains the second data is that the slave device can obtain the second data from the master according to the first identifier of the second data.
  • the device acquires the second data and copies the second data.
  • Case two The master device sends a copy data request message multiple times for different versions of data of the same name. Then, there is an overwrite action between data of different versions. That is, the first request message sent for the second data is not the first message sent by the master device to the slave device requesting data to be copied.
  • timing of the copy data request message of the different version of the data of the same name data received by the slave device is consistent with the timing of the disc down of the data of the different version of the same name data in the master device.
  • the slave device re-reads the latest version of the data of the same name on the slave device disk, determines that the version number of the latest version of the data is consistent with the version number of the data that is overwritten in the received copy data request message, and is executed by the slave device. Overwrite action, copy the data carried in the copy data request message.
  • the master device downloads the first data
  • the first data is a version of the data of the same name
  • the master device downloads the second data
  • the second data is the data of the same name
  • the first time is earlier than the second time
  • the second data is the latest version of the data of the same name.
  • the second data overwrites the first data in the master device.
  • the slave device receives the second request message at the third time and requests to obtain the first data; the slave device receives the first request message at the fourth time and requests to obtain the second data, and the third time is earlier than the fourth time.
  • the slave device receives the first request message at the fourth moment, and when the metadata of the second data is downloaded, since the slave device already stores the first data, a conflict may occur.
  • the latest version of data with the same name on the disc is read from the device, and it is determined that the latest version on the disc is the first data, and the first request message indicates that the second data overwrites the first data that has been saved in the slave device. Then, the slave device determines that the second data is data of the latest version of the data of the same name, and performs an overwrite action.
  • Case 2 for copying data in this application is a common case. That is, when the first request message instructs to acquire the second data from the device and overwrite the second data with the first data, the latest version of the data of the same name stored in the slave device is the first data, and the slave device according to the first The request message can complete the acquisition of the second data and cover the first data.
  • Case 3 The master device sends multiple copies of the data request message for data of different versions of the same name. Then, there is an overwrite action between data of different versions. That is, the first request message sent for the second data is not the first message sent by the master device to the slave device requesting data to be copied.
  • the timing of discarding data of the data of different versions of the same-named data in the master device is inconsistent.
  • the slave device downloads the metadata
  • the received copy data request message indicates that the overwritten data does not exist in the slave device.
  • the slave device cannot determine the overlay relationship between the metadata of the data to be copied carried in the received copy data request message and the data on the disc.
  • the master device downloads the first data
  • the first data is a version of the data of the same name
  • the master device downloads the second data
  • the second data is the data of the same name
  • the first time is earlier than the second time
  • the second data is the latest version of the data of the same name.
  • the second data overwrites the first data in the master device.
  • the slave device receives the second request message at the fourth time and instructs to obtain the first data; the slave device receives the first request message at the third time and instructs to obtain the second data, and the third time is earlier than the fourth time.
  • the slave device receives the first request message at the third moment.
  • the first request message indicates that the data overwritten by the second data is the first data.
  • the slave device reads the latest version of the data of the same name on the disc.
  • the fourth moment that is, at the third moment, the first data does not reach the slave device, so the first data does not exist in the slave device.
  • the slave device cannot execute the second data to overwrite the first data.
  • the version of the data of the same name stored on the slave device is the third data, where the third data is different from the above-mentioned second data and the first data, and is another version of the data of the same name.
  • the slave device cannot obtain the second data according to the first request message, and overwrite the first data. It is also impossible to determine the coverage relationship between the third data and the second data, and to copy the second data.
  • S430 is executed, and the slave device sends a second request message to the master device.
  • the slave device when the slave device receives the first request message, it cannot determine a coverage relationship between the third data and the second data. At this time, the slave device sends a second request message to the master device, where the second request message is used to query an overlay relationship between the second data and the third data in the master device.
  • step S440 the master device sends the second instruction information to the slave device.
  • the second indication information is used to indicate an overlay relationship between the second data and the third data in the master device.
  • the second indication information includes:
  • the master device sends the status of the third data and the second data in the master device to the slave device.
  • the second data is overwritten by the third data in the master device.
  • the second instruction information indicates that the second data is in a deleted state, and the third data is in a non-deleted state.
  • the second indication information includes:
  • the time when the second data and the third data are in the master device respectively. That is, the master device sends the time information of the third data and the second data in the master device to the slave device.
  • the second disk time of the second data in the master device is the first time
  • the second data time of the third data in the master device is the second time.
  • the first moment is earlier than the second moment.
  • the slave device determines an overlay relationship between the third data and the second data according to the second instruction information.
  • the coverage relationship between the third data and the second data can be determined according to the second instruction information.
  • the second instruction information indicates a state of the third data and the second data in the master device: the second data is a deleted state, and the third data is a non-deleted state. Then, the slave device updates the metadata on the slave device disk according to the second instruction information, uses the third data as the latest version of the data of the same name, and does not copy the second data;
  • the second indication information indicates a state of the third data and the second data in the master device: the third data is a deleted state, and the second data is a non-deleted state. Then, the slave device updates the metadata on the slave device disk according to the second instruction information, uses the second data as the latest version of the data of the same name, and copies the second data.
  • the second instruction information indicates the time when the third data and the second data are placed in the master device, respectively:
  • the next time of the second data is the first time
  • the second time of the third data is the second time
  • the first time is earlier than the second time. Then, the slave device determines that the third data is the latest version of the data of the same name according to the second instruction information, and the second data is not copied; or,
  • the next time of the third data is the first time
  • the second time of the second data is the second time
  • the first time is earlier than the second time.
  • the slave device updates the metadata on the slave device disk according to the second instruction information, uses the second data as the latest version of the data of the same name, and copies the second data.
  • the first request message is used to instruct the slave device to obtain the second data and cover the first data with the second data.
  • the master device and the slave device are located in different data centers, and the first data and the second data are two data of the same name. Different versions of the data, and the version of the second data is later than the version of the first data.
  • the information included in the first request message may be in the following situations:
  • the first request message includes first indication information, and the first indication information is used to indicate that a version of the second data is later than a version of the first data.
  • the slave device determines an overlay relationship between the first data and the second data according to the first instruction information.
  • the first request message further includes second data:
  • the slave device When the first request message includes the second data, the slave device receives the first request message, and first determines whether the first data covered by the second data is stored on a local disk according to the first instruction information.
  • the slave device when the latest version of the data of the same name stored on the disk of the slave device is the first data, the slave device directly receives the second data and saves the second data. Then, the first data on the slave device is overwritten according to the first instruction information, and the second data is obtained. Furthermore, the latest version of the data of the same name stored in the master device is the second data, and the latest version of the data of the same name stored in the slave device is also the second data, and the final consistency is achieved.
  • the first request message includes a first identifier of the second data, and does not include the second data:
  • the slave device When the first request message includes the first identifier of the second data, the slave device receives the first request message, and first determines whether the first data covered by the second data is stored on a local disk according to the first instruction information.
  • the slave device obtains the second data from the master device according to the first identifier of the second data in the first request message, copies the second data, and then overwrites the first data on the slave device disk according to the first instruction information. Acquire the second data. Furthermore, the latest version of the data of the same name stored in the master device is the second data, and the latest version of the data of the same name stored in the slave device is also the second data, and the final consistency is achieved.
  • the first identifier of the second data may be a version number of the second data, or other identification information capable of identifying the second data.
  • the second case the first request message does not include the foregoing first indication information.
  • the first request message includes only the second identifier of the second data.
  • the slave device can determine the coverage relationship between the first data and the second data according to the second identification of the second data.
  • the slave device determining the coverage relationship between the first data and the second data according to the second identifier of the second data includes:
  • the global scheduler allocates identifiers for different versions of data of the same name.
  • the identifier of the data can be globally unique and monotonically increasing. Sequence.
  • the second identifier of the second data When the second identifier of the second data is obtained from the device, the size of the identifier of the first data and the second identifier of the second data are compared. When the second identifier of the second data is larger than the identifier of the first data, the second data ratio is explained. The first data is updated, and it is determined that the first data is overwritten by the second data.
  • the second identifier of the second data in the second case is different from the first identifier of the second data in the first case, and the second identifier of the second data in the second case is one of the globally increasing numbers, and cannot be
  • An identifier is optionally configured for the second data; and the first identifier of the second data in the first case is only a unique identifier of the data.
  • the first request message may further include second data.
  • the latest version of the data of the same name stored on the slave device's disk is the first data
  • the latest version of the data of the same name stored in the master device is the second data
  • the latest version of the data of the same name stored in the slave device is also the second data, and the final consistency is achieved.
  • the size of the second identifier of the second data and the identifier of the third data are compared.
  • the second identifier of the data is relatively large, the second data is directly received from the device, and the second data is saved.
  • the first request message may not include the second data:
  • the slave device first according to the first data in the first request message.
  • the second identifier of the two data is obtained from the master device, and the second data is copied.
  • it is determined that the second data is newer than the first data according to the second identifier of the second data being greater than the identifier of the first data, so that the second data overwrites the first data on the disc.
  • the latest version of the data of the same name stored in the master device is the second data
  • the latest version of the data of the same name stored in the slave device is also the second data, and the final consistency is achieved.
  • the slave device obtains the second data from the master device according to the second identifier of the second data in the first request message, and copies the second data. Then, it is determined that the second data is newer than the third data according to the second identifier of the second data being greater than the identifier of the third data, so as to overwrite the third data on the disc.
  • the second identifier of the second data may be a version number of the second data, or other identification information capable of identifying the second data.
  • the first request message does not include the foregoing first indication information.
  • the first request message includes only clock information of the second data.
  • the slave device can determine the coverage relationship between the first data and the second data according to the clock information of the second data.
  • how the slave device determines the coverage relationship between the first data and the second data according to the clock information of the second data includes:
  • a global scheduler (such as a scheduling server) exists in the system, and the global scheduler allocates clock information according to the next time of the data of the same name and different versions of the data.
  • the clock information of the data is Globally unique, and one-to-one correspondence with the next time. It can be understood that the clock information of the early data of the next disk indicates the early time, and the clock information of the late data of the next disk indicates the late time. In this case, a global clock is required, and the clock information of the slave and master devices is This global clock prevails.
  • the clock information of the second data When the clock information of the second data is obtained from the device, the clock information of the first data and the clock information of the second data are compared. If the clock information of the second data is later than the clock information of the first data, the second data ratio is indicated. The first data is updated, and it is determined that the second data covers the first data.
  • the first request message may further include second data: when the latest version of the data of the same name stored on the slave device ’s disk is the first data, first receiving the second data directly from the device and saving the second data, Then, it is determined that the second data is newer than the first data according to the clock information of the second data being later than the clock information of the first data, so that the second data overwrites the first data on the disc. Furthermore, the latest version of the data of the same name stored in the master device is the second data, and the latest version of the data of the same name stored in the slave device is also the second data, and the final consistency is achieved.
  • the slave device's disk when the latest version of the data of the same name stored on the slave device's disk is not the first data, but the third data, compare the clock information of the second data with the clock information of the third data sooner or later.
  • the clock information of the data is relatively late, the second data is directly received from the device, and the second data is saved.
  • the first request message may not include the second data: when the latest version of the data of the same name stored on the slave device ’s disk is the first data, the slave device first according to the clock of the second data in the first request message Information, obtain the second data from the master device, and copy the second data. Then, it is determined that the second data is newer than the first data according to the clock information of the second data being later than the clock information of the first data, so that the second data overwrites the first data on the disc. Furthermore, the latest version of the data of the same name stored in the master device is the second data, and the latest version of the data of the same name stored in the slave device is also the second data, and the final consistency is achieved.
  • the slave device obtains the second data from the master device according to the clock information of the second data in the first request message, and copies the second data. Then, it is determined that the second data is newer than the third data according to the clock information of the second data being later than the clock information of the third data, so that the second data overwrites the third data on the disc.
  • the data of the same name also includes data of other versions, and the first data and the second data are only data of multiple versions of the data of the same name, any two of which have a direct coverage relationship.
  • the version number of the first data is a first version number
  • the version number of the second data is a second version number
  • the foregoing first indication information is carried in a header field of the first request message.
  • the first indication information is a version number of the first data.
  • the first indication information is a version number (first version number) of the first data carried in a header field of the first request message.
  • the data covered by the second data in the master device is indication information of the first data whose version number is the first version number.
  • the version number of the first data is V1
  • the version number of the second data is V2. Then, V1 is carried in the header field of the first request message.
  • the second data in the master device does not cover any data. Then, the header field of the first request message is blank.
  • the header field of the first request message is declared as a covered version number (covered_date).
  • each version of the data of the same name is identified by a unique version number.
  • the slave device sends a first response message to the master device.
  • the first response message is used to indicate that the slave device successfully copied the second data.
  • the coverage relationship between the data of each version of the same-named data is determined by the order of the data of each version, wherein the order of the metadata is modified by the CV in the metadata exclusively. control.
  • FIG. 5 is a schematic diagram of a specific embodiment of a method for copying data provided by the present application. Including user, master device, slave device and S510-S590 twelve steps.
  • the user uploads the first version of the data with the same name (referred to as the first data, and the version number is A).
  • the user uploads the second version of the data of the same name (referred to as the second data, and the version number is B).
  • the user uploads the third version of the data of the same name (referred to as the third data, and the version number is C).
  • the multiple users have uploaded three versions of data with the same name, and the multiple users can upload data with the same name at the same time.
  • the master device receives the three versions of the data with the same name.
  • the time of the data determines the sequence of the data of different versions of the same name uploaded by different users.
  • the master device sends a third copy data request message to the slave device.
  • the master device sends a copy data request message to the slave device for the three different versions of the data of the same name.
  • the third copy data request message for the third version of the data of the same name (third data) from the master device arrives at the slave device preferentially.
  • the third copy data request message carries the third data, and carries indication information indicating that the third data covers the second data in the master device.
  • the third copy data request message does not carry the third data and an identifier of the third data
  • the slave device can obtain the third data from the master device according to the identifier of the third data.
  • the slave device Due to the third copy data request message, the slave device is preferentially reached. Then, there is no second data in the slave device, and there is no data of any version of the data of the same name in the slave device, and the slave device successfully loads the third data as the latest version of the data of the same name.
  • step S521 the slave device sends a second response message to the master device, indicating that the third data is successfully copied.
  • the master device sends a second copy data request message to the slave device.
  • the second copy data request message for the second version of the data of the same name (second data) from the master device arrives at the slave device after the third copy data request message reaches the slave device.
  • the second copy data request message carries the second data, and carries instruction information indicating that the second data covers the first data in the master device.
  • the second copy data request message does not carry the second data and carries an identifier of the second data
  • the slave device can obtain the second data from the master device according to the identifier of the second data.
  • the slave device Because the first data does not exist in the slave device, and the latest version of the data with the same name exists in the slave device is the third data. At this time, the slave device cannot determine which data of the second data and the third data is the latest version of the data of the same name. S540 is executed, the slave sends a query message to the master.
  • the query message is used to query the coverage relationship between the second data and the third data in the master device.
  • S550 The master device sends third instruction information to the slave device.
  • the third instruction information indicates that the second data has been deleted in the master device, and the third data is the latest version of the data of the same name.
  • the slave device determines not to save the second data according to the third instruction information.
  • step S561 the slave device sends a third response message to the master device, indicating that the second data is successfully copied.
  • the master device sends a first copy data request message to the slave device.
  • the first copy data request message for the first version of the data of the same name (first data) from the master device arrives at the slave device after the second copy data request message reaches the slave device.
  • the first copy data request message carries the first data, and a header field of the first copy data request message is left blank. That is, the first copy data request message does not carry any coverage information.
  • the slave device receives the first copy data request message, it is determined according to the first copy data request message that the first data is the earliest version of the data of the same name, and the slave device has stored the third data. Then, the first data is deleted from the device.
  • step S571 the slave device sends a fourth response message to the master device, indicating that the first data is successfully copied.
  • the master device and the slave device in FIG. 4 and FIG. 5 may be buckets in different data centers.
  • the master device is the source bucket and the slave device is the target bucket.
  • the size of the sequence numbers of the above processes does not mean the order of execution.
  • the execution order of each process should be determined by its function and internal logic, and should not deal with the embodiments of the present application.
  • the implementation process constitutes any limitation.
  • FIG. 6 is a schematic diagram of a main device according to an embodiment of the present application.
  • the apparatus 600 shown in FIG. 6 includes a receiving unit 610, a processing unit 620, and a sending unit 630.
  • the apparatus 600 may be configured to execute steps performed by a master device in the methods shown in FIG. 4 and FIG. 5.
  • the sending unit 630 is configured to send a first request message to the slave device, where the first request message is used to instruct the slave device to obtain the second data, and overwrite the second data with the first data saved in the slave device.
  • the master device and the slave device are located in different data centers,
  • the first data and the second data are two different versions of data of the same name, and the version of the second data is later than the version of the first data.
  • the receiving unit 610 is configured to receive a first response message sent by the slave device, where the first response message is used to indicate that the slave device successfully copied the second data.
  • the first request message includes first indication information, and the first indication information is used to indicate that a version of the second data is later than a version of the first data.
  • the first indication information is carried in a header field of the first request message.
  • the first indication information is a version number of the first data.
  • the receiving unit 610 is further configured to receive a second request message sent by the slave device.
  • the second request message is used to query an overlay relationship between the second data and the third data in the master device, wherein the third data is data of a version of the data of the same name;
  • the sending unit 630 is further configured to send second instruction information to the slave device, where the second instruction information is used to indicate an overlay relationship between the second data and the third data in the master device. .
  • the second indication information includes:
  • the apparatus further includes: a processing unit 620, configured to determine a state of the second data, and the state of the second data is not overwritten. status.
  • the above device 600 may also be a master device 700.
  • the processing unit 620 may be a processor 720, and the receiving unit 610 and the sending unit 630 may be input / output interfaces 730.
  • the main device 700 may further include a memory 710 and a hard disk 740, as shown in FIG. 7.
  • FIG. 7 is a schematic block diagram of a master device according to another embodiment of the present application.
  • the master device 700 shown in FIG. 7 may include a memory 710, a processor 720, an input / output interface 730, and a hard disk 740.
  • the memory 710, the processor 720, and the input / output interface 730 and the hard disk 740 are connected through a communication connection.
  • the memory 710 is used to store program instructions
  • the processor 720 is used to execute the program instructions stored in the memory 720 to control input / output.
  • the output interface 730 receives data such as input data and information, and outputs operation results.
  • the data and information received by the input / output interface 730 may be stored in the hard disk 740.
  • the hard disk 740 is used to store data of the same name.
  • the processor 720 may use a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more Integrated circuits for executing related programs to implement the technical solutions provided in the embodiments of the present application.
  • CPU central processing unit
  • ASIC application-specific integrated circuit
  • the memory 710 may include a read-only memory and a random access memory, and provide instructions and data to the processor 720.
  • a portion of the processor 720 may also include non-volatile random access memory.
  • the processor 720 may also store information of a device type.
  • each step of the above method may be completed by an integrated logic circuit of hardware in the processor 720 or an instruction in the form of software.
  • the method disclosed in combination with the embodiments of the present application may be directly implemented by a hardware processor, or may be performed by a combination of hardware and software modules in the processor.
  • the software module may be located in a mature storage medium such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, and the like.
  • the storage medium is located in the memory 710, and the processor 720 reads information in the memory 710 and completes the steps of the foregoing method in combination with its hardware. To avoid repetition, it will not be described in detail here.
  • FIG. 8 is a schematic diagram of a slave device according to an embodiment of the present application.
  • the apparatus 800 shown in FIG. 8 includes a receiving unit 810, a processing unit 820, and a sending unit 830.
  • the apparatus 800 may be configured to perform steps performed by a slave device in the methods shown in FIG. 4 and FIG. 5.
  • the receiving unit 810 is configured to receive a first request message sent by a master device, where the first request message is used to instruct a slave device to obtain second data, and overwrite the second data with the first saved message in the slave device.
  • Data, the master device and the slave device are respectively located in different data centers,
  • the first data and the second data are two different versions of data of the same name, and the version of the second data is later than the version of the first data;
  • a processing unit 820 configured to obtain the second data according to the first request message, and overwrite the second data with the locally saved first data
  • the sending unit 830 is configured to send a first response message to the master device, where the first response message is used to indicate that the slave device successfully copied the second data.
  • the first request message includes first indication information, and the first indication information is used to indicate that a version of the second data is later than a version of the first data.
  • the first indication information is carried in a header field of the first request message.
  • the first indication information is a version number of the first data.
  • the sending unit 830 is further configured to send a second request message to the master device.
  • the second request message is used to query an overlay relationship between the second data and the third data, wherein the third data is data of a version of the data of the same name;
  • the receiving unit 810 is further configured to receive second instruction information sent by the master device, where the second instruction information is used to indicate an overlay relationship between the second data and the third data.
  • the second indication information includes:
  • the above device 800 may also be a slave device 900.
  • the processing unit 820 may be a processor 920, and the receiving unit 810 and the sending unit 830 may be input-output interfaces 930.
  • the slave device 800 may further include a memory 910 and a hard disk 940, as shown in FIG. 9.
  • FIG. 9 is a schematic block diagram of a slave device according to another embodiment of the present application.
  • the slave device 900 shown in FIG. 9 may include: a memory 910, a processor 920, an input / output interface 930, and a hard disk 940.
  • the memory 910, the processor 920, the input / output interface 930, and the hard disk 940 are connected through a communication connection.
  • the memory 910 is used to store program instructions
  • the processor 920 is used to execute the program instructions stored in the memory 920 to control input / output.
  • the output interface 930 receives data such as input data and information, and outputs operation results.
  • the data and information received by the input / output interface 930 may be stored in a hard disk 940, for example, the hard disk 940 is used to store copied data of the same name.
  • the processor 920 may use a general-purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more Integrated circuits for executing related programs to implement the technical solutions provided in the embodiments of the present application.
  • CPU central processing unit
  • ASIC application specific integrated circuit
  • the memory 910 may include a read-only memory and a random access memory, and provide instructions and data to the processor 920.
  • a portion of the processor 920 may also include non-volatile random access memory.
  • the processor 920 may also store device type information.
  • each step of the above method may be completed by using an integrated logic circuit of hardware in the processor 920 or an instruction in the form of software.
  • the method disclosed in combination with the embodiments of the present application may be directly implemented by a hardware processor, or may be performed by a combination of hardware and software modules in the processor.
  • the software module may be located in a mature storage medium such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, and the like.
  • the storage medium is located in the memory 910, and the processor 920 reads the information in the memory 910 and completes the steps of the foregoing method in combination with its hardware. To avoid repetition, it will not be described in detail here.
  • the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSPs), and special-purpose integrations. Circuits (application specific integrated circuits, ASICs), ready-made programmable gate arrays (field programmable gate arrays, FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • a general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • a hard disk (HDD) as one of the storage media of the master device and the slave device may be a solid state disk (SSD), a mechanical hard disk (Mechanical hard disk), Hybrid hard disk (solid state hard disk, SSHD), etc.
  • the disclosed systems, devices, and methods may be implemented in other ways.
  • the device embodiments described above are only schematic.
  • the division of the unit is only a logical function division.
  • multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, which may be electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objective of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each of the units may exist separately physically, or two or more units may be integrated into one unit.
  • the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.
  • the technical solution of this application is essentially a part that contributes to the existing technology or a part of the technical solution can be embodied in the form of a software product.
  • the computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in the embodiments of the present application.
  • the aforementioned storage media include: U disks, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks or compact discs and other media that can store program codes .

Abstract

A method for copying data, and a master device, and a slave device. The method comprises: a master device sending a first request message to a slave device, and instructing the slave device to obtain second data and overwrite first data stored in the slave device with the second data, wherein the master device and the slave device are located at different data centers, the first data and the second data are data of two different versions of the same name, and the version of the second data is later than the version of the first data; and the master device receiving a first reply message sent by the slave device, wherein the first reply message is used for indicating that the slave device successfully copied the second data. In the embodiment of the present application, the overwriting relationship between first data and second data can be obtained by a slave device by means of a first request message, without adding historical operation information of the data to metadata of the data, thereby reducing the size of the metadata, ensuring the final consistency of asynchronous copying, and improving the performance of copied data.

Description

用于复制数据的方法、主设备以及从设备Method for copying data, master device and slave device 技术领域Technical field
本申请涉及存储领域，并且更具体地，涉及一种用于复制数据的方法、主设备以及从设备。The present application relates to the field of storage, and more particularly, to a method for copying data, a master device, and a slave device.
背景技术Background technique
在数据复制技术的高速发展下，异步复制技术成为主流的数据复制技术。现有技术中，为了保证不同数据中心存储的同名数据，在经历主集群多线程多次覆盖以及跨地域的主集群和备集群之间的异步复制之后，能够在一定时间后，达到同名数据的最新版本数据的一致，在数据的元数据中增加元时钟(meta clock)，基于元数据中的meta clock确定同名数据的不同版本的数据之间的覆盖关系，增加了元数据的数据量，进而增加了数据复制成本，影响数据复制的性能。因此，如何减小元数据的大小，提高复制数据的性能，成为亟待解决的问题。With the rapid development of data replication technology, asynchronous replication technology has become the mainstream data replication technology. In the prior art, in order to ensure that data of the same name is stored in different data centers, after multiple coverage of the main cluster by multiple threads and asynchronous replication between the primary and standby clusters across regions, the data of the same name can be reached after a certain period of time. The latest version of the data is consistent. A meta clock is added to the metadata of the data. Based on the meta clock in the metadata, the coverage relationship between different versions of the data of the same name is determined, which increases the amount of metadata data, and further It increases the cost of data replication and affects the performance of data replication. Therefore, how to reduce the size of metadata and improve the performance of copying data has become an urgent problem.
发明内容Summary of the Invention
本申请提供一种复制数据的方法和装置，能够减小元数据的大小，提高复制数据的性能。The present application provides a method and device for copying data, which can reduce the size of metadata and improve the performance of copying data.
第一方面，提供了一种用于复制数据的方法，包括：主设备向从设备发送第一请求消息，所述第一请求消息用于指令从设备获得第二数据，并将所述第二数据覆盖第一数据，所述主设备和所述从设备分别位于不同的数据中心，所述第一数据和所述第二数据为同名数据的两个不同版本的数据，所述第二数据的版本晚于所述第一数据的版本；In a first aspect, a method for copying data is provided, including: a master device sends a first request message to a slave device, the first request message is used to instruct a slave device to obtain second data, and the second device The data covers the first data, the master device and the slave device are respectively located in different data centers, the first data and the second data are two different versions of data of the same name, and the data of the second data A version later than the version of the first data;
当所述从设备中保存的所述同名数据的最新版本为所述第一数据时，所述第一请求消息能够用于指令所述从设备获得所述第二数据，并覆盖所述第一数据，所述主设备接收所述从设备发送的第一应答消息，所述第一应答消息用于表明所述从设备成功复制所述第二数据；或者，When the latest version of the data of the same name stored in the slave device is the first data, the first request message can be used to instruct the slave device to obtain the second data and cover the first data Data, the master device receives a first response message sent by the slave device, the first response message is used to indicate that the slave device successfully copied the second data; or
当所述从设备中保存的所述同名数据的最新版本不是所述第一数据，而是第三数据时，所述主设备接收所述从设备发送的第二请求消息，所述第二请求消息用于查询在所述主设备中，所述第二数据和所述第三数据的覆盖关系，其中，所述第三数据为所述同名数据的一个版本的数据；When the latest version of the data of the same name stored in the slave device is not the first data but the third data, the master device receives a second request message sent by the slave device, and the second request The message is used to query an overlay relationship between the second data and the third data in the master device, where the third data is data of a version of the data of the same name;
所述主设备向所述从设备发送第二指示信息，所述第二指示信息用于指示在所述主设备中，所述第二数据和所述第三数据的覆盖关系，所述主设备接收所述从设备发送的第一应答消息，所述第一应答消息用于表明所述从设备成功复制所述第二数据。The master device sends second instruction information to the slave device, where the second instruction information is used to indicate an overlay relationship between the second data and the third data in the master device, and the master device Receiving a first response message sent by the slave device, where the first response message is used to indicate that the slave device successfully copied the second data.
上述复制数据可以理解为复制对象、复制文件或者复制块等。The above copy data can be understood as a copy object, a copy file, or a copy block.
根据本申请实施例提供的用于复制数据的方法，通过主设备向从设备发送的第一请求消息，进行第二数据异步复制，并根据该第一请求消息覆盖从设备中已保存的第一数据。第一数据为同名数据中旧版本的数据，而第二数据为同名数据中新版本的数据。也就是说，在主设备和从设备中每次执行复制时，均复制新版本的数据，最后能够达到最终一致性。 且第二数据的元数据中不需要增加元时钟记载操作历史，减小元数据的大小，提高复制数据的性能。According to the method for copying data provided in the embodiment of the present application, asynchronous second copy of the second data is performed through the first request message sent from the master device to the slave device, and the first saved message in the slave device is overwritten according to the first request message. data. The first data is the old version of the data with the same name, and the second data is the new version of the data with the same name. In other words, each time a copy is performed in the master device and the slave device, a new version of data is copied, and eventually consistency can be achieved. In addition, the metadata of the second data does not need to add a metadata clock to record the operation history, reduce the size of the metadata, and improve the performance of copying data.
当第一请求消息指令第二数据覆盖第一数据，而从设备中保存的同名数据的最新版本的数据不是第一数据，而保存的是第三数据时，从设备无法确定第二数据和第三数据的覆盖关系。When the first request message instructs the second data to overwrite the first data, and the latest version of the data of the same name stored in the slave is not the first data and the third data is stored, the slave cannot determine the second data and the first data. Coverage relationship of three data.
此时，增加回源校验机制，即，主设备接收从设备发送的用于查询第三数据和第二数据覆盖关系的第二请求消息，且主设备根据第三数据和第二数据在自身中的覆盖关系，向从设备发送第二指示信息，重新指示第三数据和第二数据的覆盖关系。能够保证数据复制时，主设备和从设备之间的最终一致性。At this time, a back-to-source check mechanism is added, that is, the master device receives a second request message sent by the slave device for querying the coverage relationship between the third data and the second data, and the master device In the coverage relationship in, the second instruction information is sent to the slave device, and the coverage relationship between the third data and the second data is re-indicated. Can guarantee the final consistency between the master device and the slave device when data is copied.
具体地，第二指示信息具体可以是第三数据第二数据的状态信息，或者，Specifically, the second indication information may specifically be status information of the third data and the second data, or
当机器时钟可信时，第二指示信息具体可以是第三数据和第二数据的下盘时刻。其中，时钟可信指的是主设备和从设备所在的系统设有原子时钟，对于主设备侧的时钟信息从设备侧是接受的。能够为第二指示信息的可能的指示形式提供灵活选择性。When the machine clock is reliable, the second indication information may specifically be the time when the third data and the second data are off. Among them, the clock trusted means that the system where the master device and the slave device are located is provided with an atomic clock, and the clock information on the master device side is accepted from the device side. It can provide flexibility and selectivity for possible indication forms of the second indication information.
当第二数据在主设备中被第三数据覆盖的情况下，第二数据的状态为删除(被覆盖)状态，而第三数据为覆盖第二数据的数据，则，第三数据的状态为非删除(未被覆盖)状态；由于第二数据在主设备中被第三数据覆盖，则，第二数据在主设备中的下盘时刻早于第三数据的下盘时刻。When the second data is overwritten by the third data in the master device, the state of the second data is the deleted (overwritten) state, and the third data is the data that covers the second data, then the state of the third data is Non-deleted (not overwritten) state; because the second data is overwritten by the third data in the master device, the next time of the second data in the main device is earlier than the time of the third data.
结合第一方面，在第一方面的某些实现方式中，所述第一请求消息包括第一指示信息，所述第一指示信息用于指示所述第二数据的版本晚于所述第一数据的版本。With reference to the first aspect, in some implementations of the first aspect, the first request message includes first indication information, and the first indication information is used to indicate that a version of the second data is later than the first The version of the data.
根据本申请实施例提供的用于复制数据的方法，通过在上述第一请求消息中携带所述第二数据的版本晚于所述第一数据的版本的指示信息，使得从设备在接收到第一请求消息时，能够根据其中携带的第一指示信息判断第二数据和第一数据之间的覆盖关系。提高复制数据的准确性。According to the method for copying data provided in the embodiment of the present application, by carrying indication information that the version of the second data is later than the version of the first data in the first request message, the slave device receives the first When a request message is received, the coverage relationship between the second data and the first data can be determined according to the first instruction information carried in the request message. Improve the accuracy of copied data.
具体地，上述第一请求消息还包括第二数据，或者，包括第二数据的第一标识。Specifically, the first request message further includes second data, or includes a first identifier of the second data.
当第一请求消息包括第二数据时，从设备直接接收第二数据并保存第二数据。即，从设备获得第二数据的一种实现；When the first request message includes the second data, the slave device directly receives the second data and saves the second data. That is, an implementation of obtaining the second data from the device;
当第一请求消息包括第二数据的第一标识时，从设备根据该第二数据的第一标识从主设备中获取第二数据，并复制该第二数据。即，从设备获得第二数据的另一种实现。When the first request message includes the first identifier of the second data, the slave device obtains the second data from the master device according to the first identifier of the second data, and copies the second data. That is, another implementation of obtaining the second data from the device.
具体地，第二数据的第一标识可以为第二数据的版本号，或者，其他能够指示第二数据的标识信息。Specifically, the first identification of the second data may be a version number of the second data, or other identification information capable of indicating the second data.
可选地，在一些实施例中，第一请求消息中可以不包括上述的第一指示信息，包括以下两种可能的情况：Optionally, in some embodiments, the first request message may not include the foregoing first indication information, and includes the following two possible situations:
情况一：上述主设备和从设备位于的不同的数据中心之间包括一个全局的调度器，该调度器能够为同名数据上传的每个版本的数据，分配一个全局唯一的第二标识，并且该第二标识为递增的序列，也就是说新版本的第二标识一定大于旧版本的第二标识。在此情况下，第一请求消息中可以仅仅包括第二数据的第二标识，或者，包括第二数据的第二标识和第二数据。Case 1: The above-mentioned master device and slave device include a global scheduler between different data centers. The scheduler can allocate a globally unique second identifier for each version of data uploaded with the same name data, and the The second identifier is an increasing sequence, that is, the second identifier of the new version must be greater than the second identifier of the old version. In this case, the first request message may include only the second identifier of the second data, or the second identifier and the second data of the second data.
具体地，第一请求消息中仅仅包括第二数据的第二标识时，从设备根据该第二数据的 第二标识从主设备中获取第二数据，并复制第二数据。即，从设备获得第二数据的一种实现。Specifically, when the first request message includes only the second identifier of the second data, the slave device obtains the second data from the master device according to the second identifier of the second data, and copies the second data. That is, an implementation of obtaining the second data from the device.
或者，当第一请求消息中包括第二数据的第二标识和第二数据时，从设备直接接收第二数据并保存第二数据。即，从设备获得第二数据的另一种实现情况二：上述主设备和从设备位于的不同的数据中心之间包括一个全局的调度器，该调度器能够为同名数据上传的每个版本的数据，分配一个全局唯一的时钟(系统中存在原子钟)，并且该时钟能够准确表示同名数据的每个版本的数据在主设备中的下盘时刻，也就是说新版本的时钟一定晚于旧版本的时钟。在此情况下，第一请求消息中可以仅仅包括第二数据的时钟信息，或者，包括第二数据的时钟信息和第二数据。Alternatively, when the first request message includes the second identifier of the second data and the second data, the slave device directly receives the second data and saves the second data. That is, the second implementation of obtaining the second data from the slave device: The above-mentioned master device and the slave device include a global scheduler between different data centers, and the scheduler can upload data for each version of the same name data. Data, assign a globally unique clock (there is an atomic clock in the system), and this clock can accurately represent the time when each version of the data of the same name is in the master device, that is, the new version of the clock must be later than the old version Clock. In this case, the first request message may include only the clock information of the second data, or the clock information of the second data and the second data.
具体地，第一请求消息中仅仅包括第二数据的时钟信息时，从设备根据该第二数据的时钟信息从主设备中获取第二数据，并复制第二数据。即，从设备获得第二数据的一种实现。或者，当第一请求消息中包括第二数据的时钟信息和第二数据时，从设备直接接收第二数据并保存第二数据。即，从设备获得第二数据的另一种实现。Specifically, when only the clock information of the second data is included in the first request message, the slave device obtains the second data from the master device according to the clock information of the second data, and copies the second data. That is, an implementation of obtaining the second data from the device. Alternatively, when the clock information and the second data of the second data are included in the first request message, the slave device directly receives the second data and saves the second data. That is, another implementation of obtaining the second data from the device.
结合第一方面及其上述实现方式，在第一方面的另一种实现方式中，所述第一指示信息携带在所述第一请求消息的头域。With reference to the first aspect and the foregoing implementation manners of the first aspect, in another implementation manner of the first aspect, the first indication information is carried in a header field of the first request message.
根据本申请实施例提供的用于复制数据的方法，第一指示信息可以携带在第一请求消息的头域。使得从设备接收到第一请求消息时，能够快速获得所述第二数据的版本晚于所述第一数据的版本，进而得知第一数据和第二数据的覆盖关系。According to the method for copying data provided in the embodiment of the present application, the first indication information may be carried in a header field of the first request message. When the slave device receives the first request message, it can quickly obtain the version of the second data later than the version of the first data, and then learn the coverage relationship between the first data and the second data.
可选地，第一指示信息可以携带在第一请求消息的其他位置。Optionally, the first indication information may be carried in another position of the first request message.
结合第一方面及其上述实现方式，在第一方面的另一种实现方式中，所述第一指示信息为所述第一数据的版本号。With reference to the first aspect and the foregoing implementation manners of the first aspect, in another implementation manner of the first aspect, the first indication information is a version number of the first data.
根据本申请实施例提供的用于复制数据的方法，第一指示信息可以为第一数据的版本号。由于同名数据不同版本的数据的版本号能够唯一确定该版本号对应的数据，因此，第一指示信息为第一数据的版本号，能够提高从设备确定第一数据的准确性。According to the method for copying data provided in the embodiment of the present application, the first indication information may be a version number of the first data. Since the version numbers of data of different versions of the same name data can uniquely determine the data corresponding to the version number, the first indication information is the version number of the first data, which can improve the accuracy of determining the first data from the device.
结合第一方面及其上述实现方式，在第一方面的另一种实现方式中，在所述主设备向从设备发送第一请求消息之前，所述方法还包括：所述主设备确定所述第二数据的状态，所述第二数据的状态为未被覆盖的状态。With reference to the first aspect and the foregoing implementation manners thereof, in another implementation manner of the first aspect, before the master device sends a first request message to a slave device, the method further includes: the master device determines the The state of the second data, the state of the second data is an uncovered state.
根据本申请实施例提供的用于复制数据的方法，主设备针对第二数据发送第一请求消息指令从设备获取第二数据之前，判断第二数据的状态为未被覆盖状态，有需要针对第二数据发送第一请求消息。能够避免复制信息的冗余，节省信令开销。According to the method for copying data provided by the embodiment of the present application, before the master device sends a first request message instruction to the second data to acquire the second data from the device, the master device determines that the state of the second data is an uncovered state. Two data send the first request message. It can avoid redundancy of duplicated information and save signaling overhead.
第二方面，提供了一种用于复制数据的方法，包括：从设备接收主设备发送的第一请求消息，所述第一请求消息用于指令从设备获得第二数据，并将所述第二数据覆盖第一数据，所述主设备和所述从设备分别位于不同的数据中心，所述第一数据和所述第二数据为同名数据的两个不同版本的数据，所述第二数据的版本晚于所述第一数据的版本；In a second aspect, a method for copying data is provided, including: receiving a first request message sent by a master device from a slave device, the first request message being used to instruct the slave device to obtain second data, and The two data cover the first data, the master device and the slave device are respectively located in different data centers, the first data and the second data are two different versions of data of the same name, and the second data The version of is later than the version of the first data;
当所述从设备中保存的所述同名数据的最新版本为所述第一数据时，所述从设备根据所述第一请求消息，获得所述第二数据，并将第二数据覆盖本地已保存的所述第一数据；所述从设备向所述主设备发送第一应答消息，所述第一应答消息用于表明所述从设备成功复制所述第二数据；或者，When the latest version of the data of the same name stored in the slave device is the first data, the slave device obtains the second data according to the first request message, and overwrites the second data locally The saved first data; the slave device sends a first response message to the master device, the first response message is used to indicate that the slave device successfully copied the second data; or,
当所述从设备中保存的所述同名数据的最新版本不是所述第一数据，而是第三数据时，所述从设备向所述主设备发送第二请求消息，所述第二请求消息用于查询所述第二数据和所述第三数据的覆盖关系，其中，所述第三数据为所述同名数据的一个版本的数据；所述从设备接收所述主设备发送的第二指示信息，所述第二指示信息用于指示所述第二数据和所述第三数据的覆盖关系。When the latest version of the data of the same name stored in the slave device is not the first data but the third data, the slave device sends a second request message to the master device, and the second request message For querying the coverage relationship between the second data and the third data, wherein the third data is data of one version of the data of the same name; and the slave device receives a second instruction sent by the master device Information, and the second indication information is used to indicate a coverage relationship between the second data and the third data.
根据本申请实施例提供的用于复制数据的方法，通过从设备接收主设备发送的第一请求消息，进行第二数据异步复制，并根据该第一请求消息覆盖第一数据。第一数据为同名数据中旧版本的数据，而第二数据为同名数据中新版本的数据。也就是说，在主设备和从设备中每次执行复制时，均复制新版本的数据，最后能够达到最终一致性。且第二数据的元数据中不需要增加元时钟记载操作历史，减小元数据的大小，提高复制数据的性能。According to the method for copying data provided in the embodiment of the present application, a slave device receives a first request message sent by a master device, performs asynchronous copying of the second data, and overwrites the first data according to the first request message. The first data is the old version of the data with the same name, and the second data is the new version of the data with the same name. In other words, each time a copy is performed in the master device and the slave device, a new version of data is copied, and eventually consistency can be achieved. In addition, the metadata of the second data does not need to add a metadata clock to record the operation history, reduce the size of the metadata, and improve the performance of copying data.
当第一指示信息指示第二数据覆盖第一数据，而从设备中没有保存第一数据，而保存的是第三数据时，从设备无法确定第二数据和第三数据的覆盖关系。When the first instruction information indicates that the second data covers the first data, and the slave device does not store the first data, and the slave device stores the third data, the slave device cannot determine the overlay relationship between the second data and the third data.
此时，增加回源校验机制，即，从设备向主设备发送用于查询第三数据和第二数据覆盖关系的第二请求消息，且主设备根据第三数据和第二数据在自身中的覆盖关系，向从设备发送第二指示信息，重新指示第三数据和第二数据的覆盖关系。能够保证数据复制时，主设备和从设备之间的最终一致性。At this time, a back-to-source check mechanism is added, that is, the slave device sends a second request message for querying the coverage relationship between the third data and the second data to the master device, and the master device stores the second request message in itself according to the third data and the second data. Sending the second indication information to the slave device to re-indicate the coverage relationship between the third data and the second data. Can guarantee the final consistency between the master device and the slave device when data is copied.
第二指示信息具体可以是第三数据第二数据的状态信息，或者，第三数据和第二数据的下盘时刻。能够为第二指示信息的可能的指示形式提供灵活选择性。The second indication information may specifically be status information of the third data and the second data, or a next disk time of the third data and the second data. It can provide flexibility and selectivity for possible indication forms of the second indication information.
具体地，当第二数据在主设备中被第三数据覆盖的情况下，第二数据的状态为删除(被覆盖)状态，而第三数据为覆盖第二数据的数据，则，第三数据的状态为非删除(未被覆盖)状态；由于第二数据在主设备中被第三数据覆盖，则，第二数据在主设备中的下盘时刻早于第三数据的下盘时刻。Specifically, when the second data is overwritten by the third data in the master device, the state of the second data is a deleted (overwritten) state, and the third data is the data that covers the second data, then the third data The state of is the non-deleted (not overwritten) state; because the second data is overwritten by the third data in the master device, the time at which the second data is discarded in the master device is earlier than the time at which the third data is discarded.
结合第二方面，在第二方面的某些实现方式中，所述第一请求消息包括第一指示信息，所述第一指示信息用于指示所述第二数据的版本晚于所述第一数据的版本。With reference to the second aspect, in some implementations of the second aspect, the first request message includes first indication information, and the first indication information is used to indicate that a version of the second data is later than the first data The version of the data.
根据本申请实施例提供的用于复制数据的方法，通过在上述第一请求消息中携带所述第二数据的版本晚于所述第一数据的版本的指示信息，使得从设备在接收到第一请求消息时，能够根据其中携带的第一指示信息判断第二数据和第一数据之间的覆盖关系。提高复制数据的准确性。According to the method for copying data provided in the embodiment of the present application, by carrying indication information that the version of the second data is later than the version of the first data in the first request message, the slave device receives the first When a request message is received, the coverage relationship between the second data and the first data can be determined according to the first instruction information carried in the request message. Improve the accuracy of copied data.
结合第二方面及其上述实现方式，在第二方面的另一种实现方式中，所述第一指示信息携带在所述第一请求消息的头域。With reference to the second aspect and the foregoing implementation manners thereof, in another implementation manner of the second aspect, the first indication information is carried in a header field of the first request message.
根据本申请实施例提供的用于复制数据的方法，第一指示信息可以携带在第一请求消息的头域。使得从设备接收到第一请求消息时，能够快速获得第一数据和第二数据的覆盖关系。According to the method for copying data provided in the embodiment of the present application, the first indication information may be carried in a header field of the first request message. When the slave device receives the first request message, it can quickly obtain the coverage relationship between the first data and the second data.
可选地，第一指示信息可以携带在第一请求消息的其他位置。Optionally, the first indication information may be carried in another position of the first request message.
结合第二方面及其上述实现方式，在第二方面的另一种实现方式中，所述第一指示信息为所述第一数据的版本号。With reference to the second aspect and the foregoing implementation manners thereof, in another implementation manner of the second aspect, the first indication information is a version number of the first data.
根据本申请实施例提供的用于复制数据的方法，第一指示信息可以为第一数据的版本号。由于同名数据不同版本的数据的版本号能够唯一确定该版本号对应的数据，因此，第 一指示信息为第一数据的版本号，能够提高从设备确定第一数据的准确性。According to the method for copying data provided in the embodiment of the present application, the first indication information may be a version number of the first data. Because the version numbers of data of different versions of the same name data can uniquely determine the data corresponding to the version number, the first indication information is the version number of the first data, which can improve the accuracy of determining the first data from the device.
第三方面，提供一种主设备，所述主设备包括用于执行上述第一方面及第一方面的任意可能的实现方式中的方法的各个单元。According to a third aspect, a master device is provided, and the master device includes each unit for performing the foregoing first aspect and the method in any possible implementation manner of the first aspect.
第四方面，提供一种从设备，所述从设备包括用于执行上述第二方面及第二方面的任意可能的实现方式中的方法的各个单元。According to a fourth aspect, a slave device is provided, where the slave device includes each unit for performing the foregoing second aspect and the method in any possible implementation manner of the second aspect.
第五方面，提供了一种主设备，包括至少一个处理器和至少一个存储器。该至少一个存储器用于存储计算机程序，该至少一个处理器用于从至少一个存储器中调用并运行该计算机程序，使得该主设备执行上述第一方面及第一方面的任意可能的实现方式中的方法，主设备还包括硬盘，该硬盘用于存储所述同名数据。In a fifth aspect, a host device is provided, including at least one processor and at least one memory. The at least one memory is configured to store a computer program, and the at least one processor is configured to call and run the computer program from the at least one memory, so that the main device executes the foregoing first aspect and the method in any possible implementation manner of the first aspect. The main device also includes a hard disk, which is used to store the data of the same name.
第六方面，提供了一种从设备，包括至少一个处理器和至少一个存储器。该至少一个存储器用于存储计算机程序，该至少一个处理器用于从存储器中调用并运行该计算机程序，使得该从设备执行上述第二方面及第二方面的任意可能的实现方式中的方法，该从设备还包括硬盘，该硬盘用于存储所述同名数据。According to a sixth aspect, a slave device is provided, including at least one processor and at least one memory. The at least one memory is configured to store a computer program, and the at least one processor is configured to call and run the computer program from the memory, so that the slave device executes the foregoing second aspect and the method in any possible implementation manner of the second aspect, the The slave device also includes a hard disk for storing the data of the same name.
第七方面，提供了一种系统，包括第五方面的主设备以及第六方面的从设备。In a seventh aspect, a system is provided, including a master device in the fifth aspect and a slave device in the sixth aspect.
第八方面，提供了一种计算机程序产品，所述计算机程序产品包括：计算机程序代码，当所述计算机程序代码在计算机上运行时，使得计算机执行上述第一方面和第二方面中的方法。According to an eighth aspect, a computer program product is provided. The computer program product includes computer program code that, when the computer program code runs on a computer, causes the computer to execute the methods in the first and second aspects described above.
需要说明的是，上述计算机程序代码可以全部或者部分存储在第一存储介质上，其中第一存储介质可以与处理器封装在一起的，也可以与处理器单独封装，本申请实施例对此不作具体限定。It should be noted that the above computer program code may be stored in whole or in part on a first storage medium, where the first storage medium may be packaged with the processor, or may be packaged separately with the processor. This embodiment of the present application does not deal with this. Specific limitations.
第九方面，提供了一种计算机可读介质，所述计算机可读介质存储有程序代码，当所述计算机程序代码在计算机上运行时，使得计算机执行上述第一方面和第二方面中的方法。According to a ninth aspect, a computer-readable medium is provided. The computer-readable medium stores program code, and when the computer program code runs on a computer, the computer causes the computer to execute the methods in the first and second aspects. .
本申请提出的用于复制数据的方法、主设备以及从设备，待复制的数据的元数据中不需要记载该数据的操作历史，能够减小元数据的大小，提高复制数据的性能。The method for copying data, the master device, and the slave device proposed in this application do not need to record the operation history of the data in the metadata of the data to be copied, which can reduce the size of the metadata and improve the performance of copying the data.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
图1是适用本申请实施例提供的用于复制数据的方法的场景示意图。FIG. 1 is a schematic diagram of a scenario to which a method for copying data provided by an embodiment of the present application is applied.
图2是一种基于meta clock异步复制的方法示意图。Figure 2 is a schematic diagram of a method based on metaclock asynchronous replication.
图3是另一种基于meta clock异步复制的方法示意图。Figure 3 is a schematic diagram of another asynchronous replication method based on metaclock.
图4是本申请提出的一种用于复制数据的方法示意图。FIG. 4 is a schematic diagram of a method for copying data proposed by the present application.
图5是本申请提供的一种用于复制数据的方法具体实施例的示意图。FIG. 5 is a schematic diagram of a specific embodiment of a method for copying data provided by the present application.
图6是本申请实施例的主设备的示意图。FIG. 6 is a schematic diagram of a main device according to an embodiment of the present application.
图7是本申请另一实施例的主设备的示意性框图。FIG. 7 is a schematic block diagram of a master device according to another embodiment of the present application.
图8是本申请实施例的从设备的示意图。FIG. 8 is a schematic diagram of a slave device according to an embodiment of the present application.
图9是本申请另一实施例的从设备的示意性框图。FIG. 9 is a schematic block diagram of a slave device according to another embodiment of the present application.
具体实施方式detailed description
下面将结合附图，对本申请中的技术方案进行描述。The technical solutions in this application will be described below with reference to the drawings.
本申请实施例的技术方案可以应用于不同的数据中心之间。The technical solutions in the embodiments of the present application can be applied between different data centers.
下面结合图1简单介绍本申请实施例所适用的场景。The following briefly describes the scenarios applicable to the embodiments of the present application with reference to FIG. 1.
图1是适用本申请实施例提供的用于复制数据的方法的场景示意图。包括110和120两个部分。FIG. 1 is a schematic diagram of a scenario to which a method for copying data provided by an embodiment of the present application is applied. Includes 110 and 120 parts.
110为数据中心，具体地，包括如图1所示的数据中心111和数据中心112，本申请实施例中，数据中心111和数据中心112为两个不同的数据中心。110 is a data center, and specifically includes a data center 111 and a data center 112 as shown in FIG. 1. In the embodiment of the present application, the data center 111 and the data center 112 are two different data centers.
可选地，上述数据中心111和数据中心112可以位于两个不同的区域。Optionally, the data center 111 and the data center 112 may be located in two different areas.
可选地，上述数据中心111和数据中心112可以为相同的区域中的不同的数据中心。Optionally, the data center 111 and the data center 112 may be different data centers in the same area.
可以将数据中心111中的参与数据复制流程的设备称为主设备，将数据中心112中参与数据复制流程的设备称为从设备；或者，A device participating in the data copy process in the data center 111 may be referred to as a master device, and a device participating in the data copy process in the data center 112 may be referred to as a slave device; or,
也可以将数据中心111中的参与数据复制流程的设备称为从设备，将数据中心112所中参与数据复制流程的设备称为主设备。本申请对此并不限制。The devices participating in the data replication process in the data center 111 may also be referred to as slave devices, and the devices participating in the data replication process in the data center 112 may be referred to as master devices. This application is not limited to this.
应理解，本申请实施例中对于从设备的个数也不限制，可以是在多个从设备和一个主设备之间进行数据复制。It should be understood that the number of slave devices is not limited in the embodiments of the present application, and data replication may be performed between multiple slave devices and one master device.
例如，图1中还包括多个数据中心，每个数据中心中包括一个从设备，在主设备和从设备之间进行数据复制。For example, FIG. 1 also includes multiple data centers, and each data center includes a slave device, and data is replicated between the master device and the slave device.
还应理解，上述主设备以及从设备的称呼只是一种示例，不能限制本申请的保护范围。It should also be understood that the above-mentioned names of the master device and the slave device are merely examples, and cannot limit the protection scope of the present application.
例如，主设备还可以称为主节点、主集群、主区域(region)或主端等；从设备还可以称为从节点、从集群、从区域或从端等。For example, a master device can also be called a master node, a master cluster, a master region (region), or a master end; a slave device can also be called a slave node, a slave cluster, a slave region, or a slave end, and so on.
120为复制通道，用于支持主设备和从设备之间待复制数据的传输。120 is a copy channel, and is used to support transmission of data to be copied between the master device and the slave device.
例如，由于业务原因，用户需要将数据从一个数据中心迁移至另一个数据中心，可以将迁移数据的数据中心中的迁出数据的设备称为上述主设备，将迁移数据的目的数据中心中的迁入数据的设备称为上述从设备。For example, due to business reasons, users need to migrate data from one data center to another. The device that migrates data from the data center where the data is migrated can be referred to as the above-mentioned master device. The device that migrates data is called the above-mentioned slave device.
应理解，图1中所示的主设备和从设备为集中元数据管理的最大可服务单元，例如，可以为包括处理器和存储器的网络设备或终端设备。It should be understood that the master device and the slave device shown in FIG. 1 are the largest serviceable units of centralized metadata management, and for example, may be a network device or a terminal device including a processor and a memory.
为了便于理解本申请实施例，下面简单介绍本申请实施例中将会涉及的基本概念。In order to facilitate understanding of the embodiments of the present application, the basic concepts involved in the embodiments of the present application are briefly introduced below.
异步复制(asynchronous replication)：以异步的形式开展对数据的复制动作，而不影响本地数据的下盘时延。其中，数据的下盘具体指的是：一种实现数据持久化的方式，例如，将数据从易失性的内存中存储到非易失性的硬盘上。Asynchronous replication: Asynchronous replication of data is performed without affecting the latency of the local data. Among them, the data disk specifically refers to: a way to implement data persistence, for example, storing data from a volatile memory to a non-volatile hard disk.
具体地，以异步的形式开展对数据的复制，主设备复制数据和进行数据的下盘为两个线程，互不影响。Specifically, the replication of data is performed in an asynchronous form, and the replication of data by the master device and the execution of the data are performed by two threads without affecting each other.
例如，对于同名数据A，主设备首先接收到用户上传的同名数据A的第一个版本的数据，向从设备发送数据复制请求，请求从设备复制该同名数据A的第一个版本的数据，在主设备本地，进行该同名数据A的第一个版本的数据的下盘，与第一个版本的数据的复制之间独立。For example, for the data of the same name, the master device first receives the first version of the data of the same name uploaded by the user, sends a data copy request to the slave device, and requests the slave device to copy the first version of the data of the same name. On the master device, the first disk version of the data of the same name is downloaded to the disk, and the data copy of the first version is independent.
应理解，上述主设备和从设备之间进行数据复制时，针对同名数据主设备和从设备中存储的数据希望能够达到最终一致性。It should be understood that when data is copied between the master device and the slave device, the data stored in the master device and the slave device with the same name is expected to reach the final consistency.
其中，最终一致性(final consistency)指的是：主设备和从设备中各自存储的两个同 名数据，在经历主设备和从设备中针对同名数据不同版本的数据之间的多次覆盖，以及主设备和从设备之间的异步复制，能够保证一定时间后，主设备和从设备中的同名数据的最新版本一致。Among them, final consistency refers to: two pieces of data with the same name stored in the master device and the slave device, and multiple overwrites between the master device and the slave device for data of different versions of the same name data, and Asynchronous replication between the master device and the slave device can ensure that after a certain period of time, the latest versions of the data of the same name in the master device and the slave device are consistent.
同名数据指的是数据库中有名称相同的数据。Data with the same name refers to data with the same name in the database.
例如，当向数据库里面添加新的数据，如果数据库里有与待添加的数据同名的数据，则新的数据覆盖旧的数据。For example, when adding new data to the database, if there is data in the database with the same name as the data to be added, the new data overwrites the old data.
应理解，上述数据的覆盖指的是同名数据的多个版本之间发生的覆盖，因为在非多版本的场景下，数据库中对于同名数据只能包括一个版本。而在主设备和/或从设备开启多版本时，不存在同名数据的多个版本的数据发生覆盖的情况。因为，主设备和/或从设备开启多版本时会保留同名数据的多个版本。It should be understood that the coverage of the above data refers to the coverage that occurs between multiple versions of data of the same name, because in a non-multi-version scenario, the database can only include one version of data of the same name. When multiple versions of the master device and / or the slave device are enabled, there is no case where data of multiple versions of the same name data is overwritten. Because, when the master device and / or the slave device enable multiple versions, multiple versions of the same name data are retained.
由上所述，应理解本申请中所涉及的数据复制，指的是跨数据中心的、未开启多版本场景的不同设备之间，针对同名数据的异步复制。From the above, it should be understood that the data replication involved in this application refers to the asynchronous replication of data of the same name between different devices across data centers without multi-version scenarios enabled.
进一步地，主设备和从设备可以用元时钟(meta clock)存储同名数据不同版本数据的覆盖历史。Further, the master device and the slave device may use meta clocks to store the coverage history of data of different versions with the same name.
具体地，每条meta clock会保存历史数据的创建时间(包括逻辑时钟和机器时钟)，以及当时的操作上下文。比较同名数据的两个版本的数据的两个meta clock列表，可以得出两个版本的数据的下盘时序。Specifically, each meta clock saves the creation time of the historical data (including the logical clock and the machine clock), and the operating context at that time. Comparing the two meta clock lists of the two versions of the data with the same name, we can get the timing of the next disk of the two versions of the data.
应理解，上述主设备和从设备之间能够进行数据复制，是因为在跨数据中心的两个主设备和从设备之间，拥有能够可靠传输数据的协议实现或控制机制，屏蔽数据传输本身可能带来的一致性干扰因子。It should be understood that the above-mentioned master device and slave device can copy data because the two master devices and slave devices across the data center have a protocol implementation or control mechanism capable of reliably transmitting data, and shielding the data transmission itself may Consistency interference factor.
并发事务(concurrent version，CV)：一种数据并发事务的乐观锁实现。其中，乐观锁(Optimistic Loc)是一种并发控制的方法。假设多个用户并发的事务在处理时不会彼此互相影响，各事务能够在不产生锁的情况下处理各自影响的那部分数据。在提交数据更新之前，每个事务会先检查在该事务读取数据后，有没有其他事务又修改了该数据。如果其他事务有更新的话，正在提交的事务会进行回滚。Concurrent transaction (CV): An optimistic lock implementation of data concurrent transactions. Among them, Optimistic Lock is a method of concurrent control. Assume that concurrent transactions of multiple users do not affect each other during processing, and each transaction can process the part of data that it affects without generating a lock. Before committing the data update, each transaction first checks whether any other transaction has modified the data after the transaction reads the data. If other transactions are updated, the committing transaction will be rolled back.
具体地，乐观锁，大多是基于数据版本(version)记录机制实现。数据版本指的是为数据增加一个版本标识。Specifically, optimistic locking is mostly implemented based on a data version recording mechanism. Data version refers to adding a version identifier to the data.
例如，在基于数据库表的版本解决方案中，一般是通过为数据库表增加一个“version”字段来实现。读取出数据时，将此版本号一同读出，之后更新时，对此版本号加一。此时，将提交数据的版本数据与数据库表对应记录的当前版本信息进行比对，如果提交的数据版本号大于数据库表当前版本号，则予以更新，否则认为是过期数据。For example, in a version solution based on a database table, it is generally implemented by adding a "version" field to the database table. When reading out the data, read this version number together, and when updating later, add one to this version number. At this time, the version data of the submitted data is compared with the current version information of the corresponding record of the database table. If the version number of the submitted data is greater than the current version number of the database table, it is updated, otherwise it is considered to be outdated data.
写前日志(write-ahead logging，WAL)：是关系数据库系统中用于提供原子性和持久性的一系列技术。Write-ahead logging (WAL): A series of technologies used to provide atomicity and durability in relational database systems.
在使用WAL的系统中，所有的修改在提交之前都要先写入日志(log)文件中。log文件中通常包括恢复(redo)和撤销(undo)信息。In a system using WAL, all changes are written to a log file before committing. The log file usually includes redo and undo information.
例如，假设一个程序在执行某些操作的过程中机器掉电了，在机器重新启动时，程序可能需要知道当时执行的操作是成功了还是部分成功或者是失败了。如果使用了WAL，程序就可以检查log文件，并对突然掉电时计划执行的操作内容跟实际上执行的操作内容进 行比较。在这个比较的基础上，程序就可以决定是撤销已做的操作还是继续完成已做的操作，或者是保持原样。For example, suppose a program loses power while the program is performing certain operations. When the machine restarts, the program may need to know whether the operation performed at that time was successful, partially successful, or failed. If WAL is used, the program can check the log file and compare the content of the operation planned to be performed with the actual operation during a sudden power failure. Based on this comparison, the program can decide whether to undo the operation or continue to complete the operation, or leave it as it is.
现有技术中，为了实现不同的数据中心中的主设备和从设备之间针对同名数据存储的数据的一致性，提出一种基于上述meta clock异步复制的方法。In the prior art, in order to achieve the consistency of data stored with the same name between the master device and the slave device in different data centers, a method based on the above-mentioned metaclock asynchronous replication is proposed.
具体地，meta clock包括同名数据的不同版本的数据创建时间(createTime)以及需求信息(requestInfo)等信息。Specifically, the meta clock includes information such as data creation time (createTime) and request information (requestInfo) of different versions of data of the same name.
进一步地，同名数据不同版本数据的元数据中用meta clock字段存储对这份数据的N个操作历史，比较同名数据的不同版本的数据对应的元数据中的meta clock，来决定同名数据的不同版本的数据先后覆盖关系。Further, the metadata of different versions of the same name data uses the meta clock field to store the N operation history of this data, and the meta clock in the metadata corresponding to the data of different versions of the same name data is compared to determine the difference of the data of the same name. The version of the data has successively covered the relationship.
下面，以从设备接收到同名数据的两个版本的数据的复制请求，判断该两个版本的数据的先后覆盖关系为例，说明上述的基于meta clock异步复制的方法。The following describes the method for asynchronous replication based on metaclock by taking a copy request of two versions of data of the same name data received from the device and judging the overwrite relationship between the two versions of data as an example.
具体地，从设备比较该两个版本的数据的元数据中的meta clock分为两步：Specifically, comparing the meta clock in the metadata of the two versions of data from the device is divided into two steps:
第一步：从设备用基于公共子串的方式，合并同名数据的两个版本的数据的元数据中的meta clock，同名数据的两个版本的数据的元数据中的meta clock的非公共部分按照时间排序。Step 1: The slave uses the common substring-based method to merge the meta clock in the metadata of the two versions of the data with the same name, and the non-public part of the meta clock in the metadata of the two versions of the data with the same name. Sort by time.
其中，公共子串指的是不同的子串，在各自的母串中都出现并且出现顺序与母串保持一致，称该不同的子串为不同母串的公共子串。Among them, the common substrings refer to different substrings, all of which appear in their respective parent strings and appear in the same order as the parent strings. The different substrings are called common substrings of different parent strings.
例如，子串bo,bg,lg在母串cnblogs与belong中都出现过并且出现顺序与母串保持一致，将子串bo,bg,lg称为公共子串。For example, the substrings bo, bg, lg appear in both the parent string cnblogs and belongs, and the appearance order is consistent with the parent string. The substrings bo, bg, lg are called common substrings.
第二步、从设备根据合并后的meta clock，判断，同名数据的两个版本的数据的元数据中的meta clock的先后次序。In the second step, the slave device judges, according to the merged meta clock, the order of the meta clock in the metadata of the two versions of the data of the same name.
下面结合图2简单介绍上述的基于meta clock异步复制的方法。The method of asynchronous replication based on metaclock is briefly introduced below with reference to FIG. 2.
图2是一种基于meta clock异步复制的方法示意图。包括主设备、从设备。Figure 2 is a schematic diagram of a method based on metaclock asynchronous replication. Including master and slave devices.
具体地，主设备前后上传了同名数据的多个不同版本的数据。例如，包括两个版本的数据(如图2所示，数据V9和数据V8)，数据V9覆盖了数据V8。即，主设备中同名数据的最新版本为数据V9。Specifically, the main device uploads data of multiple versions of the same name data before and after. For example, it includes two versions of data (as shown in FIG. 2, data V9 and data V8), and data V9 covers data V8. That is, the latest version of the data of the same name in the master device is data V9.
V9为数据V9的版本号，其中，图2中V9后面括号里面的数据表示数据V9的元数据中包括的meta clock对应的时间戳(1、2、3、7)。V9 is the version number of the data V9. Among them, the data in parentheses after V9 in FIG. 2 indicates the time stamp (1,2, 3, 7) corresponding to the metaclock included in the metadata of the data V9.
可以理解为，时刻“1”下盘的同名数据，被时刻“2”下盘的同名数据覆盖。同理，时刻“2”下盘的同名数据，被时刻“3”下盘的同名数据覆盖；时刻“3”下盘的同名数据，被时刻“7”下盘的同名数据覆盖。It can be understood that the data of the same name on the disk at time "1" is overwritten by the data of the same name on the disk at time "2". In the same way, the data of the same name published at time "2" is overwritten by the data of the same name published at time "3"; the data of the same name published at time "3" is overwritten by the data of the same name published at "7".
V8为数据V8的版本号，其中，图2中V8后面括号里面的数据表示数据V8的元数据中包括的meta clock对应的时间戳(2、3、4、5、6)。V8 is the version number of the data V8. The data in parentheses after V8 in FIG. 2 indicates the time stamp (2, 3, 4, 5, 6) corresponding to the metaclock included in the metadata of the data V8.
可以理解为，时刻“2”下盘的同名数据，被时刻“3”下盘的同名数据覆盖。同理，时刻“3”下盘的同名数据，被时刻“4”下盘的同名数据覆盖；时刻“4”下盘的同名数据，被时刻“5”下盘的同名数据覆盖；时刻“5”下盘的同名数据，被时刻“6”下盘的同名数据覆盖。It can be understood that the data of the same name on the disk at time "2" is overwritten by the data of the same name on the disk at time "3". In the same way, the data of the same name published at time "3" is overwritten by the data of the same name published at time "4"; the data of the same name at time "4" is overwritten by the data of the same name at time "5"; time "5" The data of the same name in the lower disk is overwritten by the data of the same name in the lower disk at "6".
在主设备和从设备之间实现异步复制的流程主要为：The process of implementing asynchronous replication between the master and slave devices is mainly:
首先，主设备针对上述数据V9，向从设备发送第一复制数据请求消息，请求从设备复 制数据V9；主设备针对上述数据V8，向从设备发送第二复制数据请求消息，请求从设备复制数据V8。First, the master device sends a first copy data request message to the slave device for the data V9, and requests the slave device to copy the data V9; the master device sends a second copy data request message to the slave device to the data V8, and requests the slave device to copy the data V8.
由上所述，由于数据V9为同名数据的新的版本的数据，因此在主设备中数据V9覆盖了数据V8。From the above, since the data V9 is a new version of the data of the same name, the data V9 covers the data V8 in the master device.
可选地，主设备向从设备发送的第一复制数据请求消息先于第二复制数据请求消息发出，并且，第一复制数据请求消息先于第二复制数据请求消息到达从设备。Optionally, the first copy data request message sent by the master device to the slave device is sent before the second copy data request message, and the first copy data request message arrives at the slave device before the second copy data request message.
或者，主设备向从设备发送的第一复制数据请求消息先于第二复制数据请求消息发出，但是，第一复制数据请求消息晚于第二复制数据请求消息到达从设备。即，主设备向从设备发送的复制数据请求消息到达从设备的时序发生交叉。在此情况下，从设备根据第一复制数据请求消息携带的数据V9的元数据中的meta clock，以及第二复制数据请求消息携带的数据V8的元数据中的meta clock，来决定数据V9和数据V8先后覆盖关系。如果，从设备不进行数据V8和数据V9的元数据中的meta clock的比较，直接根据接收第一复制数据请求消息和第二复制数据请求消息的先后关系，来确定数据V9和数据V8的覆盖关系，会导致从设备中同名数据的最新版本为数据V8。Alternatively, the first copy data request message sent by the master device to the slave device is sent before the second copy data request message, but the first copy data request message arrives later than the second copy data request message. That is, the timing at which the copy data request message sent by the master device to the slave device reaches the slave device crosses. In this case, the slave device determines the data V9 and according to the metaclock in the metadata of the data V9 carried by the first copy data request message and the metaclock in the metadata of the data V8 carried in the second copy data request message. Data V8 successively covers the relationship. If, the slave device does not compare the meta clock in the metadata of data V8 and data V9, and directly determines the coverage of data V9 and data V8 according to the sequence relationship between the first copy data request message and the second copy data request message Relationship, the latest version of the data with the same name in the slave device is data V8.
下面，详细介绍从设备如何通过比较数据V8和数据V9的元数据中的meta clock来决定数据V8和数据V9的覆盖关系。In the following, it is described in detail how the slave device determines the coverage relationship between the data V8 and the data V9 by comparing the meta clock in the metadata of the data V8 and the data V9.
首先，从设备用基于公共子串的合并方式，合并数据V8和数据V9的元数据中的meta clock。First, the slave uses the common substring-based merge method to merge the meta clock in the metadata of data V8 and data V9.
如图2所示，数据V9的元数据中的meta clock对应的时间戳为：1、3、2、7；As shown in Figure 2, the timestamp corresponding to the meta clock in the metadata of the data V9 is: 1, 3, 2, 7;
数据V8的元数据中的meta clock对应的时间戳为：3、2、4、5、6。The timestamp corresponding to the meta clock in the metadata of the data V8 is: 3, 2, 4, 5, 6.
则，二者的公共子串为：3、2，进而，首次合并结果为1、3、2。Then, the common substrings of the two are: 3, 2, and the first merge result is 1, 3, and 2.
其次，从设备将数据V8和数据V9的元数据中的meta clock的非公共部分按照时间排序。Second, the slave sorts the non-public part of the meta clock in the metadata of the data V8 and data V9 according to time.
最终结果为：1、3、2、4、5、6、7。由于时间戳“7”是数据V9的元数据中的meta clock对应的时间戳，同时，时间戳“7”是合并数据V8和数据V9的元数据中的meta clock结果中的最新时间戳。The final result is: 1, 3, 2, 4, 5, 6, 7. Since the timestamp "7" is the timestamp corresponding to the metaclock in the metadata of the data V9, and the timestamp "7" is the latest timestamp in the metaclock result of the merged data V8 and the data V9.
从而，从设备判断数据V9为同名数据的最新的版本，将数据V8覆盖。Therefore, the slave device judges that the data V9 is the latest version of the data of the same name, and overwrites the data V8.
图2所示的基于meta clock异步复制的方法。存在以下缺点：The asynchronous replication method based on meta clock shown in Figure 2. The following disadvantages exist:
一、基于meta clock异步复制的方法，需要在同名数据的每个版本的数据的元数据中增加meta clock，增加了成本。First, the asynchronous replication method based on meta clock needs to add meta clock to the metadata of each version of the data of the same name, which increases the cost.
二、基于meta clock异步复制的方法，在异步复制的过程中主设备需要预读同名数据的各个版本的数据的元数据，进而影响异步复制的性能。Second, based on the asynchronous method of meta clock, in the process of asynchronous replication, the master device needs to pre-read the metadata of each version of the data of the same name, which affects the performance of asynchronous replication.
三、基于meta clock异步复制的方法，当主设备和/或从设备存储的meta clock的个数，超过主设备和/或从设备能够存储meta clock个数的上限时，异步复制的最终一致性得不到保障。Third, based on the asynchronous method of meta clock, when the number of meta clocks stored by the master device and / or the slave device exceeds the upper limit of the number of meta clocks that can be stored by the master device and / or the slave device, the final consistency of the asynchronous replication is obtained. No guarantee.
具体地，结合图3说明上述缺点中的第三点。Specifically, the third point among the above disadvantages will be described with reference to FIG. 3.
下面结合图3简单介绍，当主设备存储的meta clock的个数，超过主设备能够存储的meta clock个数的上限时，从设备如何确定同名数据的最新的版本。The following briefly describes in conjunction with FIG. 3, when the number of meta clocks stored by the master device exceeds the upper limit of the number of meta clocks that can be stored by the master device, how the slave device determines the latest version of data with the same name.
下面结合图3简单介绍，当主设备存储的meta clock的个数超过meta clock存储个数的上限，且主设备接收同名数据的不同版本的数据发生时间跳变时，从设备如何确定同名数据的最新的版本。The following briefly describes with reference to Figure 3. When the number of meta clocks stored by the master device exceeds the upper limit of the number of meta clock storages, and when the master device receives data of different versions of data of the same name, time jumps occur, how the slave device determines the latest data of the same name. version of.
图3是另一种基于meta clock异步复制的方法示意图。包括主设备、从设备。Figure 3 is a schematic diagram of another asynchronous replication method based on metaclock. Including master and slave devices.
假设，主设备发生时间跳变，根据meta clock的原则，数据V11可能作为最老的历史版本，然后主设备依次上传了11个数据，版本为V0-V11，假设V11最后到达从设备。主设备中meta clock列表的长度限制为11。同名数据的数据历史版本为V11-V9，当数据V10到达主设备时，数据V11将从meta clock列表例剔除。Assume that the master device has a time jump. According to the principle of meta clock, data V11 may be the oldest historical version, and then the master device uploads 11 data in sequence, the versions are V0-V11. It is assumed that V11 finally reaches the slave device. The length of the meta clock list in the master device is limited to 11. The data history version of the data with the same name is V11-V9. When the data V10 reaches the master device, the data V11 will be deleted from the meta clock list example.
从设备比较各个数据的元数据中的meta clock时，发现各个数据的元数据中的meta clock没有公共子串，开始比较各个数据的时间戳，认为数据V11为最大，应该覆盖其他数据，此时从设备根据时间戳的比较能够确定数据V11为同名数据的最新的版本，最终导致数据不一致。When comparing the metaclock in the metadata of each data from the device, it was found that the metaclock in the metadata of each data does not have a common substring, and began to compare the timestamps of each data. It is considered that the data V11 is the largest and should cover other data. At this time According to the comparison of the timestamps, the slave device can determine that the data V11 is the latest version of the data of the same name, and eventually the data is inconsistent.
为了解决图2或图3中异步数据复制的最终一致性方法存在的问题。本申请提出一种用于复制数据的方法，用于上述的异步数据复制时，能够减小数据的元数据大小，实现主设备和从设备之间数据的一致性。In order to solve the problem of the eventual consistency method of asynchronous data replication in FIG. 2 or FIG. 3. This application proposes a method for copying data, which can be used to reduce the metadata size of the data and achieve the consistency of data between the master device and the slave device when used for the asynchronous data copying described above.
下面结合图4和图5详细介绍本申请中的用于复制数据的方法。能够应用于至少两个数据中心中，下面以主设备和从设备分别位于两个不同的数据中心为例，简单介绍本申请中的用于复制数据的方法。The method for copying data in the present application is described in detail below with reference to FIGS. 4 and 5. It can be applied to at least two data centers. The following uses a master device and a slave device respectively in two different data centers as examples to briefly introduce the method for copying data in this application.
图4是本申请提出的一种用于复制数据的方法示意图。包括主设备、从设备以及S410-S450五个步骤，下面详细介绍这五个步骤。FIG. 4 is a schematic diagram of a method for copying data proposed by the present application. Including the master device, slave device and S410-S450 five steps, the five steps are described in detail below.
具体地，本申请实施例中，所述的复制数据包括：对象复制、文件复制以及块复制等。Specifically, in the embodiment of the present application, the copy data includes: object copy, file copy, block copy, and the like.
可选地，在本申请实施例提供的用于复制数据的方法中，可选地，主设备和从设备之间的数据复制针对的是需要复制的数据，也就是说，主设备在向从设备发送复制数据请求消息之前，会判断该数据是否需要复制。Optionally, in the method for copying data provided in the embodiment of the present application, optionally, the data copy between the master device and the slave device is for data that needs to be copied, that is, the master device is sending data to the slave device. Before the device sends a copy data request message, it determines whether the data needs to be copied.
具体地，图4包括S410，主设备确定第二数据的状态。Specifically, FIG. 4 includes S410, and the master device determines a state of the second data.
为了降低同名数据的多个版本的数据之间大量覆盖场景下，从设备和主设备之间的信息传输次数。In order to reduce the number of data coverage scenarios between multiple versions of data with the same name, the number of times of information transmission between the slave device and the master device.
主设备在针对第二数据向从设备发送第一请求消息，指令从设备获取第二数据之前，确定所述第二数据的状态，当第二数据的状态为未被覆盖的状态时向从设备发送第一请求消息，指令从设备获取所述第二数据。其中，第二数据的为未被覆盖的状态也可以称为非删除的状态，第一请求消息也可以称为复制数据请求消息。The master device sends a first request message to the slave device for the second data and instructs the slave device to obtain the second data, and determines the state of the second data, and sends the second data to the slave device when the state of the second data is not overwritten. Send a first request message to instruct to acquire the second data from a device. The state in which the second data is not overwritten may also be referred to as a non-deleted state, and the first request message may also be referred to as a copy data request message.
下面，为了不失一般性，以主设备根据同名数据的某个版本的数据的状态，确定是否针对该版本的数据发送复制数据请求消息为例，说明主设备根据某个版本的数据的状态决定是否发送复制数据请求消息的可行性。In the following, in order not to lose the generality, the master device determines whether to send a copy data request message for the data of a certain version according to the state of the data of the same name. Whether to send a copy data request message.
其中，复制数据请求消息为上述的第一请求消息的一般性概括。应理解，上述第一请求消息为针对第二数据发送的复制数据请求消息。The copy data request message is a general summary of the foregoing first request message. It should be understood that the foregoing first request message is a copy data request message sent for the second data.
具体地，主设备根据某个数据的状态，决定是否向从设备发送针对该数据的复制数据请求消息包括以下两种情况：Specifically, the master device determines whether to send a copy data request message for the data to the slave device according to the status of the data, including the following two cases:
情况一：该数据在主设备中的状态为删除(deleted)(被覆盖)状态。即，在主设备中该数据已经被比该数据版本新的数据覆盖。主设备确定该数据的状态为deleted状态，则，主设备不再针对该数据发送复制数据请求消息。Case 1: The state of the data in the master device is a deleted (overwritten) state. That is, the data has been overwritten by data newer than the data version in the master device. If the master device determines that the status of the data is deleted, the master device no longer sends a copy data request message for the data.
例如，主设备接收到用户上传的针对同名数据A的一个版本的数据(称为第一数据)，随后，主设备又接收到用户上传的针对同名数据A的另一个版本的数据(称为第二数据)，第二数据为新的版本的数据，即，第二数据将第一数据覆盖。For example, the master device receives data uploaded by the user for one version of data A of the same name (referred to as the first data), and subsequently, the master device receives data uploaded by the user for another version of data A of the same name (referred to as the first data). Two data), the second data is a new version of the data, that is, the second data overwrites the first data.
当主设备准备向从设备发送针对第一数据的发送复制数据请求消息时，确定第一数据已经被覆盖了，处于删除状态。则，主设备不再向从设备发送针对第一数据的复制数据请求消息。When the master device is ready to send a copy data request message for the first data to the slave device, it is determined that the first data has been overwritten and is in a deleted state. Then, the master device no longer sends a copy data request message for the first data to the slave device.
情况二：该数据在主设备中的状态为非删除(未被覆盖)状态，即，在主设备中该数据还未被覆盖。主设备确定该数据的状态为非删除状态，则，主设备会针对该数据发送复制数据请求消息。Case 2: The state of the data in the master device is a non-deleted (not overwritten) state, that is, the data has not been overwritten in the master device. The master device determines that the state of the data is a non-deleted state, then the master device sends a copy data request message for the data.
例如，主设备接收到用户上传的针对同名数据A的一个版本的数据(称为第一数据)，随后，主设备又接收到用户上传的针对同名数据A的另一个版本的数据(称为第二数据)，第二数据为新的版本的数据，即，第二数据将第一数据覆盖。For example, the master device receives data uploaded by the user for one version of data A of the same name (referred to as the first data), and subsequently, the master device receives data uploaded by the user for another version of data A of the same name (referred to as the first data). Two data), the second data is a new version of the data, that is, the second data overwrites the first data.
当主设备准备向从设备发送针对第一数据的发送复制数据请求消息时，确定第一数据保存在主设备中，处于未删除状态，由于异步复制，此时第一数据还未被第二数据覆盖。则，主设备向从设备发送针对第一数据的复制数据请求消息。When the master device is ready to send a copy data request message for the first data to the slave device, it is determined that the first data is stored in the master device and is in an undeleted state. Due to asynchronous replication, the first data has not been overwritten by the second data at this time. . Then, the master device sends a copy data request message for the first data to the slave device.
应理解，在情况二所示的情况下，主设备准备向从设备发送针对第一数据的复制数据请求消息的时刻，可以是主设备接收用户上传的针对同名数据A的另一个版本的数据(称为第二数据)的时刻。即，主设备发送数据复制请求消息和接收用户上传数据为两个互不干扰的线程，可以同时进行。It should be understood that, in the case shown in the second case, the moment when the master device is ready to send a copy data request message for the first data to the slave device may be the time when the master device receives another version of data uploaded by the user for the same name data A Called the second data). That is, the master device sends the data copy request message and receives the user upload data as two threads that do not interfere with each other, and can be performed simultaneously.
具体地，主设备在确定该数据需要复制时，向从设备发送针对该数据的复制数据请求消息。Specifically, when determining that the data needs to be copied, the master device sends a copy data request message for the data to the slave device.
应理解，本申请实施例中，对于从设备的个数并不限制，主设备可以同时向多个从设备发送针对该数据的复制数据请求消息。It should be understood that, in the embodiment of the present application, the number of slave devices is not limited, and the master device may simultaneously send a copy data request message for the data to multiple slave devices.
下面，以上述第二数据处于未被覆盖的状态时，主设备请求从设备复制该第二数据为例，详细介绍本申请实施例主设备如何请求从设备复制该第二数据。In the following, when the above second data is in an uncovered state, the master device requests the slave device to copy the second data as an example, and details how the master device requests the slave device to copy the second data in the embodiment of the present application.
具体地，图4包括S420，主设备向从设备发送第一请求消息。Specifically, FIG. 4 includes S420, and the master device sends a first request message to the slave device.
当上述主设备发送第一请求消息，指令从设备获取第二数据，并将所述第二数据覆盖第一数据，包括以下几种情况：When the above master device sends a first request message, it instructs the slave device to obtain the second data and overwrite the second data with the first data, including the following situations:
情况一：上述第二数据为主设备第一次请求复制该同名数据的数据，在主设备中不存在同名数据不同版本的数据的覆盖动作。Case 1: The second data mentioned above requests the master device to copy the data of the same name data for the first time, and there is no overwriting action of data of the same name and different versions of data in the master device.
可以理解为，第二数据为同名数据的第一个版本的数据。且，在主设备中没有其他版本的数据被该第二数据覆盖。则，上述第一请求消息无需指示从设备将第二数据覆盖某个数据。It can be understood that the second data is the first version of the data with the same name. And, no other version of data in the master device is overwritten by the second data. Then, the first request message does not need to instruct the slave device to overwrite the second data with a certain data.
从设备在接收到上述第一请求消息后，根据该第一请求消息直接复制第二数据。由于，第二数据为第一个请求被复制的数据，且，该第二数据未覆盖其他版本的数据，即，在从 设备中不存在该第二数据所属的同名数据的其他版本的数据，所以，在从设备中，不会发生同名数据的不同版本的数据下盘的冲突。After receiving the first request message, the slave device directly copies the second data according to the first request message. Because the second data is the first data to be copied, and the second data does not cover other versions of the data, that is, there is no other version of the data of the same name to which the second data belongs in the slave device, Therefore, in the slave device, no conflict of data downloading with different versions of the same name data will occur.
例如，第二数据为用户向主设备上传的同名数据A的第一个版本的数据，具体地，第二数据的版本号为V2；主设备向从设备发送第一请求消息。则，从设备接收到该第一请求消息，将V2存储到同名数据A对应的元数据中，获得第二数据。For example, the second data is data of the first version of the same-named data A uploaded by the user to the master device, specifically, the version number of the second data is V2; the master device sends a first request message to the slave device. Then, the slave device receives the first request message, stores V2 into the metadata corresponding to the data A of the same name, and obtains the second data.
具体地，当第一请求消息中携带该第二数据时，从设备获得第二数据的一种可能为，直接接收第二数据并保存第二数据；或者，Specifically, when the first request message carries the second data, one possibility of obtaining the second data from the device is to directly receive the second data and save the second data; or,
当第一请求消息中未携带该第二数据，携带该第二数据的第一标识时，从设备获得第二数据的一种可能为，从设备能够根据该第二数据的第一标识从主设备中获取该第二数据，复制该第二数据。When the first request message does not carry the second data, and the first identifier of the second data is carried, one possibility that the slave device obtains the second data is that the slave device can obtain the second data from the master according to the first identifier of the second data. The device acquires the second data and copies the second data.
应理解，上述的情况一中，由于不存在不同版本之间的覆盖，从设备和主设备中保存的同名数据的数据版本一致。本申请主要研究主设备和从设备中各自存储的同名数据，在经历主设备和从设备中针对同名数据不同版本的数据之间的多次覆盖，以及主设备和从设备之间的异步复制，如何保证主设备和从设备中的同名数据的最新版本一致，对情况一不做进一步研究。It should be understood that in the first case mentioned above, because there is no overlap between different versions, the data versions of the data of the same name stored in the slave device and the master device are the same. This application mainly studies the data of the same name stored in the master device and the slave device, undergoing multiple overwrites between the master device and the slave device for data of different versions of the same name data, and the asynchronous replication between the master device and the slave device. How to ensure that the latest versions of the data of the same name in the master device and the slave device are consistent, and do not further study the situation.
情况二：主设备针对同名数据的不同版本的数据，多次发送复制数据请求消息。则，不同版本的数据之间存在覆盖动作。即，针对第二数据发送的第一请求消息，并不是主设备向从设备发送的第一次请求复制数据的消息。Case two: The master device sends a copy data request message multiple times for different versions of data of the same name. Then, there is an overwrite action between data of different versions. That is, the first request message sent for the second data is not the first message sent by the master device to the slave device requesting data to be copied.
若，从设备接收到该同名数据的不同版本的数据的复制数据请求消息的时序，跟主设备中该同名数据的不同版本的数据的下盘时序一致。If the timing of the copy data request message of the different version of the data of the same name data received by the slave device is consistent with the timing of the disc down of the data of the different version of the same name data in the master device.
由于从设备中存在该同名数据的其他版本的数据，所以，当从设备根据复制数据请求消息存储元数据时，会发生冲突。Since other versions of the data of the same name exist in the slave device, when the slave device stores the metadata according to the copy data request message, a conflict occurs.
则，从设备重新读取从设备盘上同名数据的最新版本的数据，确定该最新版本的数据的版本号与接收到的复制数据请求消息中指示被覆盖的数据的版本号一致，从设备执行覆盖动作，复制该复制数据请求消息中携带的数据。Then, the slave device re-reads the latest version of the data of the same name on the slave device disk, determines that the version number of the latest version of the data is consistent with the version number of the data that is overwritten in the received copy data request message, and is executed by the slave device. Overwrite action, copy the data carried in the copy data request message.
例如，在第一时刻，主设备下盘第一数据，该第一数据为该同名数据的一个版本的数据；在第二时刻，主设备下盘第二数据，该第二数据为该同名数据的另一个版本的数据，第一时刻早于第二时刻，则，第二数据为该同名数据的最新版本的数据。在主设备中第二数据覆盖第一数据。For example, at the first moment, the master device downloads the first data, the first data is a version of the data of the same name; at the second moment, the master device downloads the second data, the second data is the data of the same name For another version of the data, the first time is earlier than the second time, then the second data is the latest version of the data of the same name. The second data overwrites the first data in the master device.
并且，从设备在第三时刻接收到第二请求消息，请求获得第一数据；从设备在第四时刻接收到第一请求消息，请求获得第二数据，第三时刻早于第四时刻。And, the slave device receives the second request message at the third time and requests to obtain the first data; the slave device receives the first request message at the fourth time and requests to obtain the second data, and the third time is earlier than the fourth time.
进而，从设备在第四时刻接收到第一请求消息，下盘第二数据的元数据时，由于从设备中已经存储第一数据，会发生冲突。从设备读取盘上同名数据的最新版本，确定盘上最新版本为第一数据，而第一请求消息指示第二数据覆盖从设备中已保存的第一数据。则，从设备确定第二数据为该同名数据最新版本的数据，执行覆盖动作。Furthermore, the slave device receives the first request message at the fourth moment, and when the metadata of the second data is downloaded, since the slave device already stores the first data, a conflict may occur. The latest version of data with the same name on the disc is read from the device, and it is determined that the latest version on the disc is the first data, and the first request message indicates that the second data overwrites the first data that has been saved in the slave device. Then, the slave device determines that the second data is data of the latest version of the data of the same name, and performs an overwrite action.
应理解，情况二所示的为本申请中用于复制数据的方法，常见的情况。即，第一请求消息，指令从设备获取第二数据，并将第二数据覆盖第一数据时，从设备中保存的同名数据的最新版本的数据即为第一数据，从设备根据该第一请求消息能够完成第二数据的获取 以及覆盖第一数据。It should be understood that the method shown in Case 2 for copying data in this application is a common case. That is, when the first request message instructs to acquire the second data from the device and overwrite the second data with the first data, the latest version of the data of the same name stored in the slave device is the first data, and the slave device according to the first The request message can complete the acquisition of the second data and cover the first data.
情况三：主设备针对同名数据的不同版本的数据，多次发送复制数据请求消息。则，不同版本的数据之间存在覆盖动作。即，针对第二数据发送的第一请求消息，并不是主设备向从设备发送的第一次请求复制数据的消息。Case 3: The master device sends multiple copies of the data request message for data of different versions of the same name. Then, there is an overwrite action between data of different versions. That is, the first request message sent for the second data is not the first message sent by the master device to the slave device requesting data to be copied.
若，从设备接收到该同名数据的不同版本的数据的复制数据请求消息的时序，跟主设备中该同名数据的不同版本的数据的下盘时序不一致。If the slave device receives the copy data request message timing of data of different versions of the same-named data, the timing of discarding data of the data of different versions of the same-named data in the master device is inconsistent.
当从设备在下盘元数据时，接收到的复制数据请求消息中指示被覆盖的数据在从设备中不存在。此时，从设备无法判断接收到的复制数据请求消息中携带的待复制数据的元数据跟盘上数据的覆盖关系。When the slave device downloads the metadata, the received copy data request message indicates that the overwritten data does not exist in the slave device. At this time, the slave device cannot determine the overlay relationship between the metadata of the data to be copied carried in the received copy data request message and the data on the disc.
例如，在第一时刻，主设备下盘第一数据，该第一数据为该同名数据的一个版本的数据；在第二时刻，主设备下盘第二数据，该第二数据为该同名数据的另一个版本的数据，第一时刻早于第二时刻，则，第二数据为该同名数据的最新版本的数据。在主设备中第二数据覆盖第一数据。For example, at the first moment, the master device downloads the first data, the first data is a version of the data of the same name; at the second moment, the master device downloads the second data, the second data is the data of the same name For another version of the data, the first time is earlier than the second time, then the second data is the latest version of the data of the same name. The second data overwrites the first data in the master device.
并且，从设备在第四时刻接收到第二请求消息，指令获取第一数据；从设备在第三时刻接收到第一请求消息，指令获取第二数据，第三时刻早于第四时刻。In addition, the slave device receives the second request message at the fourth time and instructs to obtain the first data; the slave device receives the first request message at the third time and instructs to obtain the second data, and the third time is earlier than the fourth time.
从设备在第三时刻接收到第一请求消息，第一请求消息指示被第二数据覆盖的数据为第一数据，从设备读取盘上同名数据的最新版本的数据，由于第三时刻早于第四时刻，也就是说第三时刻，第一数据的并未到达从设备，所以从设备中不存在第一数据，此时，从设备无法执行第二数据覆盖第一数据。此时，假设从设备盘上保存的同名数据的版本为第三数据，其中，第三数据与上述的第二数据和第一数据相异，为同名数据的另一个版本的数据。The slave device receives the first request message at the third moment. The first request message indicates that the data overwritten by the second data is the first data. The slave device reads the latest version of the data of the same name on the disc. At the fourth moment, that is, at the third moment, the first data does not reach the slave device, so the first data does not exist in the slave device. At this time, the slave device cannot execute the second data to overwrite the first data. At this time, it is assumed that the version of the data of the same name stored on the slave device is the third data, where the third data is different from the above-mentioned second data and the first data, and is another version of the data of the same name.
在上述情况三下，从设备无法根据第一请求消息获取第二数据，并覆盖第一数据。也不能判断第三数据和第二数据的覆盖关系，进行第二数据的复制。此情况下，执行S430，从设备向主设备发送第二请求消息。In the third case, the slave device cannot obtain the second data according to the first request message, and overwrite the first data. It is also impossible to determine the coverage relationship between the third data and the second data, and to copy the second data. In this case, S430 is executed, and the slave device sends a second request message to the master device.
具体地，从设备接收到上述第一请求消息时，无法判断第三数据和第二数据的覆盖关系。此时，从设备向主设备发送第二请求消息，所述第二请求消息用于查询在所述主设备中，所述第二数据和所述第三数据的覆盖关系。Specifically, when the slave device receives the first request message, it cannot determine a coverage relationship between the third data and the second data. At this time, the slave device sends a second request message to the master device, where the second request message is used to query an overlay relationship between the second data and the third data in the master device.
进一步地，执行S440，主设备向从设备发送第二指示信息。Further, in step S440, the master device sends the second instruction information to the slave device.
具体地，所述第二指示信息用于指示在所述主设备中，所述第二数据和所述第三数据的覆盖关系。Specifically, the second indication information is used to indicate an overlay relationship between the second data and the third data in the master device.
可选地，在一些实施例中，所述第二指示信息包括：Optionally, in some embodiments, the second indication information includes:
所述第二数据和所述第三数据的状态的信息。即，主设备将第三数据和第二数据在主设备中的状态发送给从设备。Information of the state of the second data and the third data. That is, the master device sends the status of the third data and the second data in the master device to the slave device.
例如，第二数据在主设备中被第三数据覆盖。则，第二指示信息指示第二数据为删除状态，第三数据为非删除状态。For example, the second data is overwritten by the third data in the master device. Then, the second instruction information indicates that the second data is in a deleted state, and the third data is in a non-deleted state.
可选地，在一些实施例中，所述第二指示信息包括：Optionally, in some embodiments, the second indication information includes:
所述第二数据和所述第三数据分别在所述主设备中下盘时刻的信息。即，主设备将第三数据和第二数据在主设备中的下盘时刻信息发送给从设备。The time when the second data and the third data are in the master device, respectively. That is, the master device sends the time information of the third data and the second data in the master device to the slave device.
例如，第二数据在主设备中的下盘时刻为第一时刻，第三数据在主设备中的下盘时刻为第二时刻。且第一时刻早于第二时刻。根据该时刻信息，能够确定在主设备中第三数据覆盖了第二数据。进而，从设备根据第二指示信息确定第三数据和第二数据的覆盖关系。For example, the second disk time of the second data in the master device is the first time, and the second data time of the third data in the master device is the second time. And the first moment is earlier than the second moment. Based on the time information, it can be determined that the third data covers the second data in the master device. Further, the slave device determines an overlay relationship between the third data and the second data according to the second instruction information.
具体地，从设备接收到主设备发送的第二指示信息之后，根据第二指示信息能够确定第三数据和第二数据的覆盖关系。Specifically, after the slave device receives the second instruction information sent by the master device, the coverage relationship between the third data and the second data can be determined according to the second instruction information.
例如，第二指示信息指示第三数据和第二数据在主设备中的状态：第二数据为删除状态，第三数据为非删除状态。则，从设备根据第二指示信息更新从设备盘上的元数据，将第三数据作为同名数据的最新版本的数据，不进行第二数据的复制；For example, the second instruction information indicates a state of the third data and the second data in the master device: the second data is a deleted state, and the third data is a non-deleted state. Then, the slave device updates the metadata on the slave device disk according to the second instruction information, uses the third data as the latest version of the data of the same name, and does not copy the second data;
或者，第二指示信息指示第三数据和第二数据在主设备中的状态：第三数据为删除状态，第二数据为非删除状态。则，从设备根据第二指示信息更新从设备盘上的元数据，将第二数据作为同名数据的最新版本的数据，并复制第二数据。Alternatively, the second indication information indicates a state of the third data and the second data in the master device: the third data is a deleted state, and the second data is a non-deleted state. Then, the slave device updates the metadata on the slave device disk according to the second instruction information, uses the second data as the latest version of the data of the same name, and copies the second data.
还例如，第二指示信息指示第三数据和第二数据分别在所述主设备中下盘时刻的信息：For another example, the second instruction information indicates the time when the third data and the second data are placed in the master device, respectively:
第二数据的下盘时刻为第一时刻，第三数据的下盘时刻为第二时刻，且第一时刻早于第二时刻。则，从设备根据第二指示信息确定第三数据为同名数据的最新版本的数据，不进行第二数据的复制；或者，The next time of the second data is the first time, the second time of the third data is the second time, and the first time is earlier than the second time. Then, the slave device determines that the third data is the latest version of the data of the same name according to the second instruction information, and the second data is not copied; or,
第三数据的下盘时刻为第一时刻，第二数据的下盘时刻为第二时刻，且第一时刻早于第二时刻。则，从设备根据第二指示信息更新从设备盘上的元数据，将第二数据作为同名数据的最新版本的数据，并复制第二数据。The next time of the third data is the first time, the second time of the second data is the second time, and the first time is earlier than the second time. Then, the slave device updates the metadata on the slave device disk according to the second instruction information, uses the second data as the latest version of the data of the same name, and copies the second data.
上述第一请求消息用于指令从设备获得第二数据，并将第二数据覆盖第一数据，其中，主设备和从设备位于不同的数据中心，第一数据和第二数据为同名数据的两个不同的版本的数据，且所述第二数据的版本晚于所述第一数据的版本。The first request message is used to instruct the slave device to obtain the second data and cover the first data with the second data. The master device and the slave device are located in different data centers, and the first data and the second data are two data of the same name. Different versions of the data, and the version of the second data is later than the version of the first data.
具体地，第一请求消息中包括的信息可以为以下几种情况：Specifically, the information included in the first request message may be in the following situations:
第一种情况：第一请求消息包括第一指示信息，该第一指示信息用于指示所述第二数据的版本晚于所述第一数据的版本。在此情况下，从设备根据第一指示信息确定第一数据和第二数据的覆盖关系。First case: The first request message includes first indication information, and the first indication information is used to indicate that a version of the second data is later than a version of the first data. In this case, the slave device determines an overlay relationship between the first data and the second data according to the first instruction information.
具体地，该第一请求消息中还包括第二数据：Specifically, the first request message further includes second data:
当第一请求消息中包括第二数据时，从设备接收到第一请求消息，首先根据第一指示信息判断第二数据覆盖的第一数据是否保存在本地的盘上。When the first request message includes the second data, the slave device receives the first request message, and first determines whether the first data covered by the second data is stored on a local disk according to the first instruction information.
可选地，当从设备的盘上保存的同名数据的最新版本为第一数据时，从设备直接接收第二数据，并保存第二数据。再根据第一指示信息覆盖从设备盘上的第一数据，完成第二数据的获得。进而，主设备中保存的同名数据的最新版本的数据为第二数据，从设备中保存的同名数据的最新版本的数据也为第二数据，达到最终一致性。Optionally, when the latest version of the data of the same name stored on the disk of the slave device is the first data, the slave device directly receives the second data and saves the second data. Then, the first data on the slave device is overwritten according to the first instruction information, and the second data is obtained. Furthermore, the latest version of the data of the same name stored in the master device is the second data, and the latest version of the data of the same name stored in the slave device is also the second data, and the final consistency is achieved.
可选地，当从设备的盘上保存的同名数据的最新版本不是第一数据，而是第三数据时，参照S420中的情况三，这里不再赘述。Optionally, when the latest version of the data of the same name stored on the slave device's disk is not the first data, but the third data, refer to Case 3 in S420, and details are not described herein again.
具体地，该第一请求消息中包括第二数据的第一标识，且不包括第二数据：Specifically, the first request message includes a first identifier of the second data, and does not include the second data:
当第一请求消息中包括第二数据的第一标识时，从设备接收到第一请求消息，首先根据第一指示信息判断第二数据覆盖的第一数据是否保存在本地的盘上。When the first request message includes the first identifier of the second data, the slave device receives the first request message, and first determines whether the first data covered by the second data is stored on a local disk according to the first instruction information.
可选地，当从设备的盘上保存的同名数据的最新版本为第一数据时。首先，从设备根 据第一请求消息中的第二数据的第一标识，从主设备中获取第二数据，复制该第二数据，再根据第一指示信息覆盖从设备盘上的第一数据，完成第二数据的获得。进而，主设备中保存的同名数据的最新版本的数据为第二数据，从设备中保存的同名数据的最新版本的数据也为第二数据，达到最终一致性。Optionally, when the latest version of the data of the same name saved on the disk of the slave device is the first data. First, the slave device obtains the second data from the master device according to the first identifier of the second data in the first request message, copies the second data, and then overwrites the first data on the slave device disk according to the first instruction information. Acquire the second data. Furthermore, the latest version of the data of the same name stored in the master device is the second data, and the latest version of the data of the same name stored in the slave device is also the second data, and the final consistency is achieved.
可选地，当从设备的盘上保存的同名数据的最新版本不是第一数据，而是第三数据时，参照S420中的情况三，这里不再赘述。Optionally, when the latest version of the data of the same name stored on the slave device's disk is not the first data, but the third data, refer to Case 3 in S420, and details are not described herein again.
具体地，上述第二数据的第一标识可以为第二数据的版本号，或者，为能够标识第二数据的其他标识信息。Specifically, the first identifier of the second data may be a version number of the second data, or other identification information capable of identifying the second data.
第二种情况：第一请求消息不包括上述第一指示信息。第一请求消息中仅仅包括第二数据的第二标识。在此情况下，从设备根据第二数据的第二标识能够确定第一数据和第二数据的覆盖关系。The second case: the first request message does not include the foregoing first indication information. The first request message includes only the second identifier of the second data. In this case, the slave device can determine the coverage relationship between the first data and the second data according to the second identification of the second data.
具体地，从设备根据第二数据的第二标识判断第一数据和第二数据的覆盖关系包括：Specifically, the slave device determining the coverage relationship between the first data and the second data according to the second identifier of the second data includes:
将上述多个数据中心的集合看成一个系统，系统中存在一个全局调度器(例如调度服务器)，该全局调度器为同名数据不同版本的数据分配标识，数据的标识可以为全局唯一且单调递增的数列。Regarding the collection of the above multiple data centers as a system, there is a global scheduler (such as a scheduling server) in the system. The global scheduler allocates identifiers for different versions of data of the same name. The identifier of the data can be globally unique and monotonically increasing. Sequence.
当从设备获取第二数据的第二标识时，比较第一数据的标识和第二数据的第二标识的大小，当第二数据的第二标识大于第一数据的标识，说明第二数据比第一数据更新，确定由第二数据覆盖第一数据。When the second identifier of the second data is obtained from the device, the size of the identifier of the first data and the second identifier of the second data are compared. When the second identifier of the second data is larger than the identifier of the first data, the second data ratio is explained. The first data is updated, and it is determined that the first data is overwritten by the second data.
应理解，第二种情况中第二数据的第二标识与第一种情况中第二数据的第一标识不同，第二种情况中第二数据的第二标识为全局递增数列中一个，不能随意为第二数据配置标识；而第一种情况中第二数据的第一标识仅仅为数据的唯一的标识。It should be understood that the second identifier of the second data in the second case is different from the first identifier of the second data in the first case, and the second identifier of the second data in the second case is one of the globally increasing numbers, and cannot be An identifier is optionally configured for the second data; and the first identifier of the second data in the first case is only a unique identifier of the data.
具体地，该第一请求消息中可以还包括第二数据。，当从设备的盘上保存的同名数据的最新版本为第一数据时，首先从设备直接接收第二数据，并保存第二数据，再根据第二数据的第二标识大于第一数据的标识确定第二数据比第一数据更新，从而将第二数据覆盖盘上的第一数据。进而，主设备中保存的同名数据的最新版本的数据为第二数据，从设备中保存的同名数据的最新版本的数据也为第二数据，达到最终一致性。Specifically, the first request message may further include second data. When the latest version of the data of the same name stored on the slave device's disk is the first data, first receive the second data directly from the device and save the second data, and then according to the second identifier of the second data is greater than the identifier of the first data It is determined that the second data is newer than the first data, so that the second data overwrites the first data on the disc. Furthermore, the latest version of the data of the same name stored in the master device is the second data, and the latest version of the data of the same name stored in the slave device is also the second data, and the final consistency is achieved.
可选地，当从设备的盘上保存的同名数据的最新版本为不是第一数据，而是第三数据时，比较第二数据的第二标识与第三数据的标识的大小，若第二数据的第二标识较大，从设备直接接收第二数据，并保存第二数据。Optionally, when the latest version of the data of the same name stored on the slave device's disk is not the first data but the third data, the size of the second identifier of the second data and the identifier of the third data are compared. The second identifier of the data is relatively large, the second data is directly received from the device, and the second data is saved.
具体地，该第一请求消息中可以不包括第二数据：可选地，当从设备的盘上保存的同名数据的最新版本为第一数据时，首先从设备根据第一请求消息中的第二数据的第二标识，从主设备中获取第二数据，并复制该第二数据。再根据第二数据的第二标识大于第一数据的标识确定第二数据比第一数据更新，从而将第二数据覆盖盘上的第一数据。进而，主设备中保存的同名数据的最新版本的数据为第二数据，从设备中保存的同名数据的最新版本的数据也为第二数据，达到最终一致性。Specifically, the first request message may not include the second data: Optionally, when the latest version of the data of the same name stored on the disk of the slave device is the first data, the slave device first according to the first data in the first request message. The second identifier of the two data is obtained from the master device, and the second data is copied. Then, it is determined that the second data is newer than the first data according to the second identifier of the second data being greater than the identifier of the first data, so that the second data overwrites the first data on the disc. Furthermore, the latest version of the data of the same name stored in the master device is the second data, and the latest version of the data of the same name stored in the slave device is also the second data, and the final consistency is achieved.
可选地，当从设备的盘上保存的同名数据的最新版本为不是第一数据，而是第三数据时，比较第二数据的第二标识与第三数据的标识的大小，若第二数据的第二标识较大，首先从设备根据第一请求消息中的第二数据的第二标识，从主设备中获取第二数据，并复制 该第二数据。再根据第二数据的第二标识大于第三数据的标识确定第二数据比第三数据更新，从而将第二数据覆盖盘上的第三一数据。Optionally, when the latest version of the data of the same name stored on the slave device's disk is not the first data but the third data, the size of the second identifier of the second data and the identifier of the third data are compared. The second identifier of the data is large. First, the slave device obtains the second data from the master device according to the second identifier of the second data in the first request message, and copies the second data. Then, it is determined that the second data is newer than the third data according to the second identifier of the second data being greater than the identifier of the third data, so as to overwrite the third data on the disc.
具体地，上述第二数据的第二标识可以为第二数据的版本号，或者，为能够标识第二数据的其他标识信息。Specifically, the second identifier of the second data may be a version number of the second data, or other identification information capable of identifying the second data.
第三种情况：第一请求消息不包括上述第一指示信息。第一请求消息中仅仅包括第二数据的时钟信息。在此情况下，从设备根据第二数据的时钟信息能够确定第一数据和第二数据的覆盖关系。Third case: the first request message does not include the foregoing first indication information. The first request message includes only clock information of the second data. In this case, the slave device can determine the coverage relationship between the first data and the second data according to the clock information of the second data.
具体地，从设备如何根据第二数据的时钟信息判断第一数据和第二数据的覆盖关系包括：Specifically, how the slave device determines the coverage relationship between the first data and the second data according to the clock information of the second data includes:
将上述多个数据中心的集合看成一个系统，系统中存在一个全局调度器(例如调度服务器)，该全局调度器根据同名数据不同版本的数据的下盘时刻分配时钟信息，数据的时钟信息为全局唯一的，且与下盘时刻先后一一对应。可以理解为下盘早的数据的时钟信息指示早的时刻，下盘晚的数据的时钟信息指示晚的时刻，在此情况下，需要一个全局的时钟，从设备和主设备的时钟信息均以该全局时钟为准。Regarding the collection of the above multiple data centers as a system, a global scheduler (such as a scheduling server) exists in the system, and the global scheduler allocates clock information according to the next time of the data of the same name and different versions of the data. The clock information of the data is Globally unique, and one-to-one correspondence with the next time. It can be understood that the clock information of the early data of the next disk indicates the early time, and the clock information of the late data of the next disk indicates the late time. In this case, a global clock is required, and the clock information of the slave and master devices is This global clock prevails.
当从设备获取第二数据的时钟信息时，比较第一数据的时钟信息和第二数据的时钟信息的早晚，如果第二数据的时钟信息晚于第一数据的时钟信息，说明第二数据比第一数据更新，确定第二数据覆盖第一数据。When the clock information of the second data is obtained from the device, the clock information of the first data and the clock information of the second data are compared. If the clock information of the second data is later than the clock information of the first data, the second data ratio is indicated. The first data is updated, and it is determined that the second data covers the first data.
具体地，该第一请求消息中可以还包括第二数据：当从设备的盘上保存的同名数据的最新版本为第一数据时，首先从设备直接接收第二数据，并保存第二数据，再根据第二数据的时钟信息晚于第一数据的时钟信息确定第二数据比第一数据更新，从而将第二数据覆盖盘上的第一数据。进而，主设备中保存的同名数据的最新版本的数据为第二数据，从设备中保存的同名数据的最新版本的数据也为第二数据，达到最终一致性。Specifically, the first request message may further include second data: when the latest version of the data of the same name stored on the slave device ’s disk is the first data, first receiving the second data directly from the device and saving the second data, Then, it is determined that the second data is newer than the first data according to the clock information of the second data being later than the clock information of the first data, so that the second data overwrites the first data on the disc. Furthermore, the latest version of the data of the same name stored in the master device is the second data, and the latest version of the data of the same name stored in the slave device is also the second data, and the final consistency is achieved.
可选地，当从设备的盘上保存的同名数据的最新版本为不是第一数据，而是第三数据时，比较第二数据的时钟信息与第三数据的时钟信息的早晚，若第二数据的时钟信息较晚，从设备直接接收第二数据，并保存第二数据。Optionally, when the latest version of the data of the same name stored on the slave device's disk is not the first data, but the third data, compare the clock information of the second data with the clock information of the third data sooner or later. The clock information of the data is relatively late, the second data is directly received from the device, and the second data is saved.
具体地，该第一请求消息中可以不包括第二数据：当从设备的盘上保存的同名数据的最新版本为第一数据时，首先从设备根据第一请求消息中的第二数据的时钟信息，从主设备中获取第二数据，并复制该第二数据。再根据第二数据的时钟信息晚于第一数据的时钟信息确定第二数据比第一数据更新，从而将第二数据覆盖盘上的第一数据。进而，主设备中保存的同名数据的最新版本的数据为第二数据，从设备中保存的同名数据的最新版本的数据也为第二数据，达到最终一致性。Specifically, the first request message may not include the second data: when the latest version of the data of the same name stored on the slave device ’s disk is the first data, the slave device first according to the clock of the second data in the first request message Information, obtain the second data from the master device, and copy the second data. Then, it is determined that the second data is newer than the first data according to the clock information of the second data being later than the clock information of the first data, so that the second data overwrites the first data on the disc. Furthermore, the latest version of the data of the same name stored in the master device is the second data, and the latest version of the data of the same name stored in the slave device is also the second data, and the final consistency is achieved.
可选地，当从设备的盘上保存的同名数据的最新版本为不是第一数据，而是第三数据时，比较第二数据的时钟信息与第三数据的时钟信息的早晚，若第二数据的时钟信息较晚，首先从设备根据第一请求消息中的第二数据的时钟信息，从主设备中获取第二数据，并复制该第二数据。再根据第二数据的时钟信息晚于第三数据的时钟信息确定第二数据比第三数据更新，从而将第二数据覆盖盘上的第三数据。Optionally, when the latest version of the data of the same name stored on the slave device's disk is not the first data, but the third data, compare the clock information of the second data with the clock information of the third data sooner or later. The clock information of the data is relatively late. First, the slave device obtains the second data from the master device according to the clock information of the second data in the first request message, and copies the second data. Then, it is determined that the second data is newer than the third data according to the clock information of the second data being later than the clock information of the third data, so that the second data overwrites the third data on the disc.
下面，主要介绍第一请求消息为上述第一种情况下的所示的消息时，主设备和从设备之间的信令交互。The following mainly describes the signaling interaction between the master device and the slave device when the first request message is the message shown in the first case.
应理解，上述同名数据还包括其他版本的数据，上述第一数据和第二数据只是同名数据的多个版本的数据中，任意两个具有直接覆盖关系的数据。It should be understood that the data of the same name also includes data of other versions, and the first data and the second data are only data of multiple versions of the data of the same name, any two of which have a direct coverage relationship.
具体地，上述第一数据的版本号为第一版本号，第二数据的版本号为第二版本号。Specifically, the version number of the first data is a first version number, and the version number of the second data is a second version number.
可选地，在一些实施例中，上述第一指示信息携带在所述第一请求消息的头域。Optionally, in some embodiments, the foregoing first indication information is carried in a header field of the first request message.
可选地，在另一些实施例中，第一指示信息为所述第一数据的版本号。Optionally, in other embodiments, the first indication information is a version number of the first data.
例如，第一指示信息为在第一请求消息的头域中携带第一数据的版本号(第一版本号)。为从设备提供，在主设备中第二数据覆盖的数据为版本号为第一版本号的第一数据的指示信息。For example, the first indication information is a version number (first version number) of the first data carried in a header field of the first request message. Provided for the slave device, the data covered by the second data in the master device is indication information of the first data whose version number is the first version number.
具体地，上述第一数据的版本号为V1，第二数据的版本号为V2。则，第一请求消息的头域中携带V1。Specifically, the version number of the first data is V1, and the version number of the second data is V2. Then, V1 is carried in the header field of the first request message.
可选地，在一些实施例中，在主设备中上述第二数据并没有覆盖任何的数据。则，上述第一请求消息的头域置空。Optionally, in some embodiments, the second data in the master device does not cover any data. Then, the header field of the first request message is blank.
具体地，上述第一请求消息的头域声明为覆盖版本号(covered_date)。Specifically, the header field of the first request message is declared as a covered version number (covered_date).
应理解，同名数据的每一个版本的数据都使用唯一的版本号标识。It should be understood that each version of the data of the same name is identified by a unique version number.
S450，从设备向主设备发送第一应答消息。S450. The slave device sends a first response message to the master device.
具体地，所述第一应答消息用于表明所述从设备成功复制所述第二数据。Specifically, the first response message is used to indicate that the slave device successfully copied the second data.
下面结合一个具体地的实施例，说明上述用于复制数据的方法。The following describes the above method for copying data in combination with a specific embodiment.
具体地，在该实施例中，同名数据的各个版本的数据之间的覆盖关系由各个版本的数据的下盘先后决定，其中，元数据的下盘先后靠独占式的修改元数据里的CV控制。Specifically, in this embodiment, the coverage relationship between the data of each version of the same-named data is determined by the order of the data of each version, wherein the order of the metadata is modified by the CV in the metadata exclusively. control.
图5是本申请提供的一种用于复制数据的方法具体实施例的示意图。包括用户、主设备、从设备以及S510-S590十二个步骤。FIG. 5 is a schematic diagram of a specific embodiment of a method for copying data provided by the present application. Including user, master device, slave device and S510-S590 twelve steps.
S510，用户上传同名数据的第一个版本的数据(称为第一数据，版本号为A)。S510. The user uploads the first version of the data with the same name (referred to as the first data, and the version number is A).
S511，用户上传同名数据的第二个版本的数据(称为第二数据，版本号为B)。S511. The user uploads the second version of the data of the same name (referred to as the second data, and the version number is B).
S512，用户上传同名数据的第三个版本的数据(称为第三数据，版本号为C)。S512. The user uploads the third version of the data of the same name (referred to as the third data, and the version number is C).
应理解，执行上述S510、S511以及S512的用户可以为同一个用户，也可以为不同的用户。It should be understood that the users performing the above S510, S511, and S512 may be the same user or different users.
当上述用户同一个用户时，可以理解为该用户连续上传了同名数据的三个版本的数据；When the above users are the same user, it can be understood that the user has uploaded three versions of data of the same name data continuously;
当上述用户为不同的多个用户时，可以理解为该多个用户分别上传了同名数据的三个版本的数据，该多个用户可以同时上传同名数据，主设备根据接收同名数据的三个版本的数据的时间确定不同用户上传的同名数据的不同版本的数据的先后关系。When the above users are different multiple users, it can be understood that the multiple users have uploaded three versions of data with the same name, and the multiple users can upload data with the same name at the same time. The master device receives the three versions of the data with the same name. The time of the data determines the sequence of the data of different versions of the same name uploaded by different users.
S520，主设备向从设备发送第三复制数据请求消息。S520. The master device sends a third copy data request message to the slave device.
具体地，在本实施例中，假设主设备针对上述同名数据的三个不同的版本的数据均向从设备发送复制数据请求消息。Specifically, in this embodiment, it is assumed that the master device sends a copy data request message to the slave device for the three different versions of the data of the same name.
由于线程调度或者网络状态等不可控的因素。导致。主设备针对上述同名数据的第三个版本的数据(第三数据)的第三复制数据请求消息，优先到达从设备。Due to uncontrollable factors such as thread scheduling or network status. resulting in. The third copy data request message for the third version of the data of the same name (third data) from the master device arrives at the slave device preferentially.
可选地，上述第三复制数据请求消息中携带该第三数据，且携带指示信息，指示第三数据在主设备中覆盖了第二数据。Optionally, the third copy data request message carries the third data, and carries indication information indicating that the third data covers the second data in the master device.
可选地，上述第三复制数据请求消息中未携带该第三数据，携带第三数据的标识，从 设备根据该第三数据的标识能够从主设备中获取第三数据。Optionally, the third copy data request message does not carry the third data and an identifier of the third data, and the slave device can obtain the third data from the master device according to the identifier of the third data.
由于第三复制数据请求消息，优先到达从设备。则，从设备中不存在第二数据，且从设备中没有该同名数据任何版本的数据，从设备将第三数据作为同名数据的最新版本下盘成功。Due to the third copy data request message, the slave device is preferentially reached. Then, there is no second data in the slave device, and there is no data of any version of the data of the same name in the slave device, and the slave device successfully loads the third data as the latest version of the data of the same name.
进一步地，执行S521，从设备向主设备发送第二应答消息，指示复制第三数据成功。Further, in step S521, the slave device sends a second response message to the master device, indicating that the third data is successfully copied.
S530，主设备向从设备发送第二复制数据请求消息。S530. The master device sends a second copy data request message to the slave device.
由于线程调度或者网络状态等不可控的因素。导致。主设备针对上述同名数据的第二个版本的数据(第二数据)的第二复制数据请求消息，在上述第三复制数据请求消息到达从设备之后到达从设备。Due to uncontrollable factors such as thread scheduling or network status. resulting in. The second copy data request message for the second version of the data of the same name (second data) from the master device arrives at the slave device after the third copy data request message reaches the slave device.
可选地，上述第二复制数据请求消息中携带该第二数据，且携带指示信息，指示第二数据在主设备中覆盖了第一数据。Optionally, the second copy data request message carries the second data, and carries instruction information indicating that the second data covers the first data in the master device.
可选地，上述第二复制数据请求消息中未携带该第二数据，携带第二数据的标识，从设备根据该第二数据的标识能够从主设备中获取第二数据。Optionally, the second copy data request message does not carry the second data and carries an identifier of the second data, and the slave device can obtain the second data from the master device according to the identifier of the second data.
由于，从设备中不存在第一数据，且在从设备中存在该同名数据的最新版本为第三数据。此时，从设备无法判断第二数据和第三数据哪个数据为同名数据的最新版本。执行S540，从设备向主设备发送查询消息。Because the first data does not exist in the slave device, and the latest version of the data with the same name exists in the slave device is the third data. At this time, the slave device cannot determine which data of the second data and the third data is the latest version of the data of the same name. S540 is executed, the slave sends a query message to the master.
该查询消息用于查询第二数据和第三数据在主设备中的覆盖关系。The query message is used to query the coverage relationship between the second data and the third data in the master device.
S550，主设备向从设备发送第三指示信息。S550: The master device sends third instruction information to the slave device.
该第三指示信息指示第二数据在主设备中已经被删除，第三数据为同名数据的最新版本的数据。The third instruction information indicates that the second data has been deleted in the master device, and the third data is the latest version of the data of the same name.
S560，从设备根据第三指示信息确定不保存第二数据。S560. The slave device determines not to save the second data according to the third instruction information.
进一步地，执行S561，从设备向主设备发送第三应答消息，指示复制第二数据成功。Further, in step S561, the slave device sends a third response message to the master device, indicating that the second data is successfully copied.
S570，主设备向从设备发送第一复制数据请求消息。S570. The master device sends a first copy data request message to the slave device.
由于线程调度或者网络状态等不可控的因素。导致。主设备针对上述同名数据的第一个版本的数据(第一数据)的第一复制数据请求消息，在上述第二复制数据请求消息到达从设备之后到达从设备。Due to uncontrollable factors such as thread scheduling or network status. resulting in. The first copy data request message for the first version of the data of the same name (first data) from the master device arrives at the slave device after the second copy data request message reaches the slave device.
应理解，上述第一复制数据请求消息中携带该第一数据，且该第一复制数据请求消息的头域置空。即，该第一复制数据请求消息未携带任何覆盖信息。从设备接收到该第一复制数据请求消息时，根据该第一复制数据请求消息确定第一数据为同名数据的最早的版本，而且，从设备中已经保存有第三数据。则，从设备删除第一数据。It should be understood that the first copy data request message carries the first data, and a header field of the first copy data request message is left blank. That is, the first copy data request message does not carry any coverage information. When the slave device receives the first copy data request message, it is determined according to the first copy data request message that the first data is the earliest version of the data of the same name, and the slave device has stored the third data. Then, the first data is deleted from the device.
进一步地，执行S571，从设备向主设备发送第四应答消息，指示复制第一数据成功。Further, in step S571, the slave device sends a fourth response message to the master device, indicating that the first data is successfully copied.
具体地，图4和图5中的主设备和从设备可以是不同数据中心中的存储桶。主设备为源存储桶，从设备为目标存储桶。Specifically, the master device and the slave device in FIG. 4 and FIG. 5 may be buckets in different data centers. The master device is the source bucket and the slave device is the target bucket.
应理解，在本申请的各种实施例中，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。It should be understood that, in the various embodiments of the present application, the size of the sequence numbers of the above processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not deal with the embodiments of the present application. The implementation process constitutes any limitation.
下面结合图6-图9介绍本申请实施例提供的主设备和从设备。The following describes the master device and the slave device provided in the embodiments of the present application with reference to FIGS. 6 to 9.
图6是本申请实施例主设备的示意图。图6所示的装置600包括接收单元610、处理 单元620和发送单元630。其中，装置600可以用于执行图4和图5所示的方法中主设备执行的步骤。FIG. 6 is a schematic diagram of a main device according to an embodiment of the present application. The apparatus 600 shown in FIG. 6 includes a receiving unit 610, a processing unit 620, and a sending unit 630. The apparatus 600 may be configured to execute steps performed by a master device in the methods shown in FIG. 4 and FIG. 5.
发送单元630，用于向从设备发送第一请求消息，所述第一请求消息用于指令从设备获得第二数据，并将所述第二数据覆盖从设备中已保存的第一数据，所述主设备和所述从设备分别位于不同的数据中心，The sending unit 630 is configured to send a first request message to the slave device, where the first request message is used to instruct the slave device to obtain the second data, and overwrite the second data with the first data saved in the slave device. The master device and the slave device are located in different data centers,
所述第一数据和所述第二数据为同名数据的两个不同版本的数据，所述第二数据的版本晚于所述第一数据的版本。The first data and the second data are two different versions of data of the same name, and the version of the second data is later than the version of the first data.
接收单元610，用于接收所述从设备发送的第一应答消息，所述第一应答消息用于表明所述从设备成功复制所述第二数据。The receiving unit 610 is configured to receive a first response message sent by the slave device, where the first response message is used to indicate that the slave device successfully copied the second data.
具体地，所述第一请求消息中包括第一指示信息，所述第一指示信息用于指示第二数据的版本晚于所述第一数据的版本。Specifically, the first request message includes first indication information, and the first indication information is used to indicate that a version of the second data is later than a version of the first data.
可选地，所述第一指示信息携带在所述第一请求消息的头域。Optionally, the first indication information is carried in a header field of the first request message.
可选地，所述第一指示信息为所述第一数据的版本号。Optionally, the first indication information is a version number of the first data.
具体地，当所述从设备中保存的同名数据的最新版本的数据不是第一数据，而是第三数据时，所述接收单元610，还用于接收所述从设备发送的第二请求消息，所述第二请求消息用于查询在所述主设备中，所述第二数据和所述第三数据的覆盖关系，其中，所述第三数据为所述同名数据的一个版本的数据；Specifically, when the latest version of the data of the same name stored in the slave device is not the first data but the third data, the receiving unit 610 is further configured to receive a second request message sent by the slave device. The second request message is used to query an overlay relationship between the second data and the third data in the master device, wherein the third data is data of a version of the data of the same name;
所述发送单元630，还用于向所述从设备发送第二指示信息，所述第二指示信息用于指示在所述主设备中，所述第二数据和所述第三数据的覆盖关系。The sending unit 630 is further configured to send second instruction information to the slave device, where the second instruction information is used to indicate an overlay relationship between the second data and the third data in the master device. .
具体地，所述第二指示信息包括：Specifically, the second indication information includes:
所述第二数据和所述第三数据的状态的信息；或者，Information about the state of the second data and the third data; or
所述第二数据和所述第三数据分别在所述主设备中下盘时刻的信息。The time when the second data and the third data are in the master device, respectively.
具体地，在发送单元630向从设备发送第一请求消息之前，所述装置还包括：处理单元620，用于确定所述第二数据的状态，所述第二数据的状态为未被覆盖的状态。Specifically, before the sending unit 630 sends the first request message to the slave device, the apparatus further includes: a processing unit 620, configured to determine a state of the second data, and the state of the second data is not overwritten. status.
在可选的实施例中，上述装置600还可以是主设备700，具体地，所述处理单元620可以为处理器720，所述接收单元610以及发送单元630可以为输入输出接口730。所述主设备700还可以包括存储器710和硬盘740，具体如图7所示。In an optional embodiment, the above device 600 may also be a master device 700. Specifically, the processing unit 620 may be a processor 720, and the receiving unit 610 and the sending unit 630 may be input / output interfaces 730. The main device 700 may further include a memory 710 and a hard disk 740, as shown in FIG. 7.
图7是本申请另一实施例的主设备的示意性框图。图7所示的主设备700可以包括：存储器710、处理器720、输入/输出接口730和硬盘740。其中，存储器710、处理器720和输入/输出接口730和硬盘740通过通信连接相连，该存储器710用于存储程序指令，该处理器720用于执行该存储器720存储的程序指令，以控制输入/输出接口730接收输入的数据和信息，输出操作结果等数据，输入/输出接口730接收的数据和信息可以存储在硬盘740中，例如，硬盘740用于存储同名数据。FIG. 7 is a schematic block diagram of a master device according to another embodiment of the present application. The master device 700 shown in FIG. 7 may include a memory 710, a processor 720, an input / output interface 730, and a hard disk 740. The memory 710, the processor 720, and the input / output interface 730 and the hard disk 740 are connected through a communication connection. The memory 710 is used to store program instructions, and the processor 720 is used to execute the program instructions stored in the memory 720 to control input / output. The output interface 730 receives data such as input data and information, and outputs operation results. The data and information received by the input / output interface 730 may be stored in the hard disk 740. For example, the hard disk 740 is used to store data of the same name.
应理解，在本申请实施例中，该处理器720可以采用通用的中央处理器(Central Processing Unit，CPU)，微处理器，应用专用集成电路(Application Specific Integrated Circuit，ASIC)，或者一个或多个集成电路，用于执行相关程序，以实现本申请实施例所提供的技术方案。It should be understood that, in the embodiment of the present application, the processor 720 may use a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more Integrated circuits for executing related programs to implement the technical solutions provided in the embodiments of the present application.
该存储器710可以包括只读存储器和随机存取存储器，并向处理器720提供指令和数 据。处理器720的一部分还可以包括非易失性随机存取存储器。例如，处理器720还可以存储设备类型的信息。The memory 710 may include a read-only memory and a random access memory, and provide instructions and data to the processor 720. A portion of the processor 720 may also include non-volatile random access memory. For example, the processor 720 may also store information of a device type.
在实现过程中，上述方法的各步骤可以通过处理器720中的硬件的集成逻辑电路或者软件形式的指令完成。结合本申请实施例所公开的方法可以直接体现为硬件处理器执行完成，或者用处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器710，处理器720读取存储器710中的信息，结合其硬件完成上述方法的步骤。为避免重复，这里不再详细描述。In the implementation process, each step of the above method may be completed by an integrated logic circuit of hardware in the processor 720 or an instruction in the form of software. The method disclosed in combination with the embodiments of the present application may be directly implemented by a hardware processor, or may be performed by a combination of hardware and software modules in the processor. The software module may be located in a mature storage medium such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, and the like. The storage medium is located in the memory 710, and the processor 720 reads information in the memory 710 and completes the steps of the foregoing method in combination with its hardware. To avoid repetition, it will not be described in detail here.
图8是本申请实施例的从设备的示意图。图8所示的装置800包括接收单元810、处理单元820和发送单元830。其中，装置800可以用于执行图4和图5所示的方法中从设备执行的步骤。FIG. 8 is a schematic diagram of a slave device according to an embodiment of the present application. The apparatus 800 shown in FIG. 8 includes a receiving unit 810, a processing unit 820, and a sending unit 830. The apparatus 800 may be configured to perform steps performed by a slave device in the methods shown in FIG. 4 and FIG. 5.
接收单元810，用于接收主设备发送的第一请求消息，所述第一请求消息用于指令从设备获得第二数据，并将所述第二数据覆盖所述从设备中已保存的第一数据，所述主设备和所述从设备分别位于不同的数据中心，The receiving unit 810 is configured to receive a first request message sent by a master device, where the first request message is used to instruct a slave device to obtain second data, and overwrite the second data with the first saved message in the slave device. Data, the master device and the slave device are respectively located in different data centers,
所述第一数据和所述第二数据为同名数据的两个不同版本的数据，且所述第二数据的版本晚于所述第一数据的版本；The first data and the second data are two different versions of data of the same name, and the version of the second data is later than the version of the first data;
处理单元820，用于根据所述第一请求消息，获得所述第二数据，并将第二数据覆盖本地已保存的第一数据；A processing unit 820, configured to obtain the second data according to the first request message, and overwrite the second data with the locally saved first data;
发送单元830，用于向所述主设备发送第一应答消息，所述第一应答消息用于表明所述从设备成功复制所述第二数据。The sending unit 830 is configured to send a first response message to the master device, where the first response message is used to indicate that the slave device successfully copied the second data.
具体地，所述第一请求消息包括第一指示信息，所述第一指示信息用于指示第二数据的版本晚于所述第一数据的版本。Specifically, the first request message includes first indication information, and the first indication information is used to indicate that a version of the second data is later than a version of the first data.
可选地，所述第一指示信息携带在所述第一请求消息的头域。Optionally, the first indication information is carried in a header field of the first request message.
可选地，所述第一指示信息为所述第一数据的版本号。Optionally, the first indication information is a version number of the first data.
可选地，当所述从设备中保存的同名数据的最新版本的数据不是第一数据，而是第三数据时，所述发送单元830，还用于向所述主设备发送第二请求消息，所述第二请求消息用于查询所述第二数据和所述第三数据的覆盖关系，其中，所述第三数据为所述同名数据的一个版本的数据；Optionally, when the latest version of the data of the same name stored in the slave device is not the first data but the third data, the sending unit 830 is further configured to send a second request message to the master device. The second request message is used to query an overlay relationship between the second data and the third data, wherein the third data is data of a version of the data of the same name;
所述接收单元810，还用于接收所述主设备发送的第二指示信息，所述第二指示信息用于指示所述第二数据和所述第三数据的覆盖关系。The receiving unit 810 is further configured to receive second instruction information sent by the master device, where the second instruction information is used to indicate an overlay relationship between the second data and the third data.
具体地，所述第二指示信息包括：Specifically, the second indication information includes:
所述第二数据和所述第三数据的状态的信息；或者，Information about the state of the second data and the third data; or
所述第二数据和所述第三数据分别在所述主设备中下盘时刻的信息。The time when the second data and the third data are in the master device, respectively.
在可选的实施例中，上述装置800还可以是从设备900，具体地，所述处理单元820可以为处理器920，所述接收单元810以及发送单元830可以为输入输出接口930。所述从设备800还可以包括存储器910和硬盘940，具体如图9所示。In an optional embodiment, the above device 800 may also be a slave device 900. Specifically, the processing unit 820 may be a processor 920, and the receiving unit 810 and the sending unit 830 may be input-output interfaces 930. The slave device 800 may further include a memory 910 and a hard disk 940, as shown in FIG. 9.
图9是本申请另一实施例的从设备的示意性框图。图9所示的从设备900可以包括：存储器910、处理器920、输入/输出接口930和硬盘940。其中，存储器910、处理器920、 输入/输出接口930和硬盘940通过通信连接相连，该存储器910用于存储程序指令，该处理器920用于执行该存储器920存储的程序指令，以控制输入/输出接口930接收输入的数据和信息，输出操作结果等数据，输入/输出接口930接收的数据和信息可以存储在硬盘940中，例如，硬盘940用于存储复制的同名数据。FIG. 9 is a schematic block diagram of a slave device according to another embodiment of the present application. The slave device 900 shown in FIG. 9 may include: a memory 910, a processor 920, an input / output interface 930, and a hard disk 940. The memory 910, the processor 920, the input / output interface 930, and the hard disk 940 are connected through a communication connection. The memory 910 is used to store program instructions, and the processor 920 is used to execute the program instructions stored in the memory 920 to control input / output. The output interface 930 receives data such as input data and information, and outputs operation results. The data and information received by the input / output interface 930 may be stored in a hard disk 940, for example, the hard disk 940 is used to store copied data of the same name.
应理解，在本申请实施例中，该处理器920可以采用通用的中央处理器(Central Processing Unit，CPU)，微处理器，应用专用集成电路(Application Specific Integrated Circuit，ASIC)，或者一个或多个集成电路，用于执行相关程序，以实现本申请实施例所提供的技术方案。It should be understood that, in the embodiments of the present application, the processor 920 may use a general-purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more Integrated circuits for executing related programs to implement the technical solutions provided in the embodiments of the present application.
该存储器910可以包括只读存储器和随机存取存储器，并向处理器920提供指令和数据。处理器920的一部分还可以包括非易失性随机存取存储器。例如，处理器920还可以存储设备类型的信息。The memory 910 may include a read-only memory and a random access memory, and provide instructions and data to the processor 920. A portion of the processor 920 may also include non-volatile random access memory. For example, the processor 920 may also store device type information.
在实现过程中，上述方法的各步骤可以通过处理器920中的硬件的集成逻辑电路或者软件形式的指令完成。结合本申请实施例所公开的方法可以直接体现为硬件处理器执行完成，或者用处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器910，处理器920读取存储器910中的信息，结合其硬件完成上述方法的步骤。为避免重复，这里不再详细描述。In the implementation process, each step of the above method may be completed by using an integrated logic circuit of hardware in the processor 920 or an instruction in the form of software. The method disclosed in combination with the embodiments of the present application may be directly implemented by a hardware processor, or may be performed by a combination of hardware and software modules in the processor. The software module may be located in a mature storage medium such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, and the like. The storage medium is located in the memory 910, and the processor 920 reads the information in the memory 910 and completes the steps of the foregoing method in combination with its hardware. To avoid repetition, it will not be described in detail here.
应理解，本申请实施例中，该处理器可以为中央处理单元(central processing unit，CPU)，该处理器还可以是其它通用处理器、数字信号处理器(digital signal processor，DSP)、专用集成电路(application specific integrated circuit，ASIC)、现成可编程门阵列(field programmable gate array，FPGA)或者其它可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。It should be understood that, in the embodiment of the present application, the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSPs), and special-purpose integrations. Circuits (application specific integrated circuits, ASICs), ready-made programmable gate arrays (field programmable gate arrays, FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
还应理解，本申请实施例中，硬盘(hard disk drive，HDD)作为主设备和从设备的存储媒介之一，可以是固态硬盘(solid state disk，SSD)、机械硬盘(mechanical hard disk)、混合硬盘(solid state hard disk，SSHD)等。It should also be understood that in the embodiments of the present application, a hard disk (HDD) as one of the storage media of the master device and the slave device may be a solid state disk (SSD), a mechanical hard disk (Mechanical hard disk), Hybrid hard disk (solid state hard disk, SSHD), etc.
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。Those of ordinary skill in the art may realize that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments, and are not repeated here.
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods may be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, which may be electrical, mechanical or other forms.
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objective of the solution of this embodiment.
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each of the units may exist separately physically, or two or more units may be integrated into one unit.
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(read-only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application is essentially a part that contributes to the existing technology or a part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in the embodiments of the present application. The aforementioned storage media include: U disks, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks or compact discs and other media that can store program codes .
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。The above is only a specific implementation of this application, but the scope of protection of this application is not limited to this. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in this application. It should be covered by the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims (16)

  1. 一种用于复制数据的方法，其特征在于，包括：A method for copying data, comprising:
    主设备向从设备发送第一请求消息，所述第一请求消息用于指令从设备获得第二数据，并将所述第二数据覆盖所述从设备中已保存的第一数据，所述主设备和所述从设备分别位于不同的数据中心，所述第一数据和所述第二数据为同名数据的两个不同版本的数据，所述第二数据的版本晚于所述第一数据的版本；The master device sends a first request message to the slave device, the first request message is used to instruct the slave device to obtain the second data, and the second data overwrites the first data saved in the slave device, and the master device The device and the slave device are respectively located in different data centers, the first data and the second data are two different versions of data of the same name, and the version of the second data is later than that of the first data. version;
    所述主设备接收所述从设备发送的第一应答消息，所述第一应答消息用于表明所述从设备成功复制所述第二数据。The master device receives a first response message sent by the slave device, and the first response message is used to indicate that the slave device successfully copied the second data.
  2. 根据权利要求1所述的方法，其特征在于，所述第一请求消息中包括第一指示信息，所述第一指示信息用于指示所述第二数据的版本晚于所述第一数据的版本。The method according to claim 1, wherein the first request message includes first indication information, and the first indication information is used to indicate that a version of the second data is later than a version of the first data. version.
  3. 根据权利要求2所述的方法，其特征在于，所述第一指示信息为所述第一数据的版本号。The method according to claim 2, wherein the first indication information is a version number of the first data.
  4. 根据权利要求1-3中任一项所述的方法，其特征在于，在所述主设备向从设备发送第一请求消息之前，所述方法还包括：The method according to any one of claims 1-3, wherein before the master device sends a first request message to a slave device, the method further comprises:
    所述主设备确定所述第二数据的状态，所述第二数据的状态为未被覆盖的状态。The master device determines a state of the second data, and the state of the second data is a state that is not overwritten.
  5. 一种用于复制数据的方法，其特征在于，包括：A method for copying data, comprising:
    从设备接收主设备发送的第一请求消息，所述第一请求消息用于指令从设备获得第二数据，并将所述第二数据覆盖所述从设备中已保存的第一数据，所述主设备和所述从设备分别位于不同的数据中心，The slave device receives a first request message sent by the master device, the first request message is used to instruct the slave device to obtain the second data, and the second data overwrites the first data saved in the slave device; The master device and the slave device are located in different data centers,
    所述第一数据和所述第二数据为同名数据的两个不同版本的数据，所述第二数据的版本晚于所述第一数据的版本；The first data and the second data are two different versions of data of the same name, and the version of the second data is later than the version of the first data;
    所述从设备根据所述第一请求消息，获得所述第二数据，并将第二数据覆盖本地已保存的所述第一数据；Obtaining, by the slave device, the second data according to the first request message, and overwriting the second data with the first data that is stored locally;
    所述从设备向所述主设备发送第一应答消息，所述第一应答消息用于表明所述从设备成功复制所述第二数据。The slave device sends a first response message to the master device, where the first response message is used to indicate that the slave device successfully copied the second data.
  6. 根据权利要求5所述的方法，其特征在于，所述第一请求消息中包括第一指示信息，所述第一指示信息用于指示所述第二数据的版本晚于所述第一数据的版本。The method according to claim 5, wherein the first request message includes first indication information, and the first indication information is used to indicate that a version of the second data is later than that of the first data. version.
  7. 根据权利要求6所述的方法，其特征在于，所述第一指示信息为所述第一数据的版本号。The method according to claim 6, wherein the first indication information is a version number of the first data.
  8. 一种主设备，其特征在于，包括：A master device, comprising:
    发送单元，用于向从设备发送第一请求消息，所述第一请求消息用于指令从设备获得第二数据，并将所述第二数据覆盖所述从设备中已保存的第一数据，所述主设备和所述从设备分别位于不同的数据中心，A sending unit, configured to send a first request message to the slave device, where the first request message is used to instruct the slave device to obtain the second data, and overwrite the second data with the first data saved in the slave device, The master device and the slave device are respectively located in different data centers,
    所述第一数据和所述第二数据为同名数据的两个不同版本的数据，所述第二数据的版本晚于所述第一数据的版本；The first data and the second data are two different versions of data of the same name, and the version of the second data is later than the version of the first data;
    接收单元，用于接收所述从设备发送的第一应答消息，所述第一应答消息用于表明所述从设备成功复制所述第二数据。The receiving unit is configured to receive a first response message sent by the slave device, where the first response message is used to indicate that the slave device successfully copied the second data.
  9. 根据权利要求8所述的主设备，其特征在于，所述第一请求消息中包括第一指示信息，所述第一指示信息用于指示所述第二数据的版本晚于所述第一数据的版本。The master device according to claim 8, wherein the first request message includes first indication information, and the first indication information is used to indicate that a version of the second data is later than the first data version of.
  10. 根据权利要求9所述的主设备，其特征在于，所述第一指示信息为所述第一数据的版本号。The master device according to claim 9, wherein the first indication information is a version number of the first data.
  11. 根据权利要求8-10中任一项所述的主设备，其特征在于，在所述发送单元向从设备发送第一请求消息之前，所述主设备还包括：The master device according to any one of claims 8 to 10, wherein before the sending unit sends the first request message to the slave device, the master device further comprises:
    处理单元，用于确定所述第二数据的状态，所述第二数据的状态为未被覆盖的状态。A processing unit is configured to determine a state of the second data, and a state of the second data is an uncovered state.
  12. 一种从设备，其特征在于，包括：A slave device, comprising:
    接收单元，用于接收主设备发送的第一请求消息，所述第一请求消息用于指令从设备获得第二数据，并将所述第二数据覆盖所述从设备中已保存的第一数据，所述主设备和所述从设备分别位于不同的数据中心，A receiving unit, configured to receive a first request message sent by a master device, where the first request message is used to instruct a slave device to obtain second data, and overwrite the second data with the saved first data in the slave device , The master device and the slave device are located in different data centers,
    所述第一数据和所述第二数据为同名数据的两个不同版本的数据，所述第二数据的版本晚于所述第一数据的版本；The first data and the second data are two different versions of data of the same name, and the version of the second data is later than the version of the first data;
    处理单元，用于根据所述第一请求消息，获得所述第二数据，并将第二数据覆盖本地已保存的所述第一数据；A processing unit, configured to obtain the second data according to the first request message, and overwrite the second data with the first data stored locally;
    发送单元，用于向所述主设备发送第一应答消息，所述第一应答消息用于表明所述从设备成功复制所述第二数据。The sending unit is configured to send a first response message to the master device, where the first response message is used to indicate that the slave device successfully copied the second data.
  13. 根据权利要求12所述的从设备，其特征在于，所述第一请求消息包括第一指示信息，所述第一指示信息用于指示所述第二数据的版本晚于所述第一数据的版本。The slave device according to claim 12, wherein the first request message includes first indication information, and the first indication information is used to indicate that a version of the second data is later than a version of the first data. version.
  14. 根据权利要求13所述的从设备，其特征在于，所述第一指示信息为所述第一数据的版本号。The slave device according to claim 13, wherein the first indication information is a version number of the first data.
  15. 一种主设备，其特征在于，包括至少一个处理器和至少一个存储器，所述至少一个存储器用于存储计算机程序，所述至少一个处理器用于从至少一个存储器中调用并运行该计算机程序，使得该主设备执行如权利要求1-4中任一项所述的方法；A main device, comprising at least one processor and at least one memory, the at least one memory is used to store a computer program, and the at least one processor is used to call and run the computer program from the at least one memory, so that The master device executes the method according to any one of claims 1-4;
    所述主设备还包括硬盘，所述硬盘用于存储所述同名数据。The master device further includes a hard disk, and the hard disk is used to store the data of the same name.
  16. 一种从设备，其特征在于，包括至少一个处理器和至少一个存储器，所述至少一个存储器用于存储计算机程序，所述至少一个处理器用于从至少一个存储器中调用并运行该计算机程序，使得该从设备执行如权利要求5-7所述的方法；A slave device includes at least one processor and at least one memory, where the at least one memory is used to store a computer program, and the at least one processor is used to call and run the computer program from the at least one memory such that The slave device executes the method according to claims 5-7;
    所述从设备还包括硬盘，所述硬盘用于存储所述同名数据。The slave device further includes a hard disk, and the hard disk is used to store the data of the same name.
PCT/CN2019/098307 2018-08-27 2019-07-30 Method for copying data, and master device, and slave device WO2020042852A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810982126.1 2018-08-27
CN201810982126.1A CN109164985B (en) 2018-08-27 2018-08-27 Method for copying data, master device and slave device

Publications (1)

Publication Number Publication Date
WO2020042852A1 true WO2020042852A1 (en) 2020-03-05

Family

ID=64896793

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/098307 WO2020042852A1 (en) 2018-08-27 2019-07-30 Method for copying data, and master device, and slave device

Country Status (2)

Country Link
CN (1) CN109164985B (en)
WO (1) WO2020042852A1 (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109164985B (en) * 2018-08-27 2020-07-07 华为技术有限公司 Method for copying data, master device and slave device
CN110493338B (en) * 2019-08-20 2022-09-13 深圳柚石物联技术有限公司 Equipment mutual control method, system and computer readable storage medium
CN114077448A (en) * 2020-08-11 2022-02-22 深圳云天励飞技术股份有限公司 Data management method and related equipment

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103875229A (en) * 2013-12-02 2014-06-18 华为技术有限公司 Asynchronous replication method, device and system
CN104461774A (en) * 2014-11-24 2015-03-25 华为技术有限公司 Asynchronous replication method, device and system
CN109164985A (en) * 2018-08-27 2019-01-08 华为技术有限公司 For the method for replicate data, main equipment and from equipment

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7587564B2 (en) * 2006-09-26 2009-09-08 International Business Machines Corporation System, method and computer program product for managing data versions
US9720620B1 (en) * 2014-03-11 2017-08-01 Amazon Technologies, Inc. Efficient data volume replication for block-based storage
US9830333B1 (en) * 2014-06-27 2017-11-28 Amazon Technologies, Inc. Deterministic data replication with conflict resolution
CN105843702B (en) * 2015-01-14 2019-04-12 阿里巴巴集团控股有限公司 A kind of method and device for data backup
CN106528338B (en) * 2016-10-28 2020-08-14 华为技术有限公司 Remote data copying method, storage device and storage system

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103875229A (en) * 2013-12-02 2014-06-18 华为技术有限公司 Asynchronous replication method, device and system
CN104461774A (en) * 2014-11-24 2015-03-25 华为技术有限公司 Asynchronous replication method, device and system
CN109164985A (en) * 2018-08-27 2019-01-08 华为技术有限公司 For the method for replicate data, main equipment and from equipment

Also Published As

Publication number Publication date
CN109164985A (en) 2019-01-08
CN109164985B (en) 2020-07-07

Similar Documents

Publication Publication Date Title
US11397709B2 (en) Automated configuration of log-coordinated storage groups
US10262002B2 (en) Consistent execution of partial queries in hybrid DBMS
CN110502507B (en) Management system, method, equipment and storage medium of distributed database
CN109739935B (en) Data reading method and device, electronic equipment and storage medium
US11016944B2 (en) Transferring objects between different storage devices based on timestamps
US10831720B2 (en) Cloud storage distributed file system
CN107644030B (en) Distributed database data synchronization method, related device and system
CN109710388B (en) Data reading method and device, electronic equipment and storage medium
US10817498B2 (en) Distributed transactions in cloud storage with hierarchical namespace
US20190370362A1 (en) Multi-protocol cloud storage for big data and analytics
US8392482B1 (en) Versioning of database partition maps
CN107148617B (en) Automatic configuration of log-coordinated storage groups
WO2020042852A1 (en) Method for copying data, and master device, and slave device
CN111190935B (en) Data reading method and device, computer equipment and storage medium
US11748215B2 (en) Log management method, server, and database system
CN109783578B (en) Data reading method and device, electronic equipment and storage medium
CN109902127B (en) Historical state data processing method and device, computer equipment and storage medium
EP4276651A1 (en) Log execution method and apparatus, and computer device and storage medium
US11875178B2 (en) Using multiple blockchains for applying transactions to a set of persistent data objects in persistent storage systems
WO2016127557A1 (en) Method for re-establishing standby database, and apparatus thereof
US10579604B2 (en) Database system, information processing device, method and medium
CN111459913B (en) Capacity expansion method and device of distributed database and electronic equipment
US10572503B2 (en) Database system, information processing device, method and medium
US8705537B1 (en) Eventually-consistent data stream consolidation
CN114846458A (en) Distributed serializable concurrency control scheme

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19856017

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19856017

Country of ref document: EP

Kind code of ref document: A1