CN109992447A - Data copy method, device and storage medium - Google Patents
Data copy method, device and storage medium Download PDFInfo
- Publication number
- CN109992447A CN109992447A CN201711482591.0A CN201711482591A CN109992447A CN 109992447 A CN109992447 A CN 109992447A CN 201711482591 A CN201711482591 A CN 201711482591A CN 109992447 A CN109992447 A CN 109992447A
- Authority
- CN
- China
- Prior art keywords
- data
- node
- slot
- data slot
- request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1464—Management of the backup or restore process for networked environments
Abstract
This application discloses a kind of data copy method, device and storage mediums, belong to data processing field.This method is applied to data copy system, the data copy system includes client and N number of node, this method comprises: i-th of node in N number of node receives i-th of data slot in N number of data slot of the client transmitted in parallel, which is that the client is obtained according to N number of node to sent the progress cutting of destination request data；It gives i-th of data slot transmitted in parallel to other nodes, and receives other data slots that other nodes are sent respectively；I-th of data slot and other data slots are stored, to complete the duplication of the destination request data.The application can be improved data copying efficiency and system performance, and the problem of can become performance bottleneck to avoid Leader node.
Description
Technical field
This application involves data processing field, in particular to a kind of data copy method, device and storage medium.
Background technique
Data duplication is a kind of data redundancy backup method, can be during service operation by multiple node standbies
Data, when some nodes break down, by the service switching of the node to other nodes, to continue to take over by other nodes
Business, to guarantee the reliabilty and availability of system.Data copy method is applied particularly in data copy system, and the data are multiple
System processed includes client and multiple nodes, and client is used to send out the client for sending request data, multiple node
The request data sent is replicated.
Currently, most common data copy method is leader follower replication method, in leader follower replication method, data are replicated
Multiple nodes that system includes can set main side (Leader) node for some node in multiple node in advance, and
Standby host end (Follower) node is set by other nodes.In data reproduction process, client can first be sent out request data
Leader node is given, so that the request data received is written in partitioned file by Leader node.Each Follower
Node could pull the number of request after request data is written to partitioned file by Leader node from Leader node
According to, and the request data is written in the partitioned file of itself, it can send and be written to Leader node after being written successfully
Success response.When Leader node receive all Follower nodes in multiple node transmission write-in success response it
Afterwards, it can confirm that the request data replicates successfully.
In leader follower replication method, request data is replicated due to needing to first pass through Leader node, then waits each
Follower node respectively replicates request data, therefore duplicating efficiency is lower, and performance is poor.Moreover, because ask the visitor in for institute
The duplication of data is asked to require to first pass through Leader node, therefore when data volume is more than the ability to bear of Leader node,
Leader node will become the bottleneck of data duplication.
Summary of the invention
This application provides a kind of data copy method, device and storage medium, can be used for solving depositing in the related technology
Duplicating efficiency it is low, system performance is poor and the bottleneck problem of Leader node.The technical solution is as follows:
In a first aspect, providing a kind of data copy method, the method is applied to data copy system, and the data are multiple
System processed includes client and N number of node, which comprises
I-th of node in N number of node receives i-th in N number of data slot of the client transmitted in parallel
Data slot, N number of data slot are that the client is carried out according to N number of node to sent destination request data
Cutting obtains, and the i is greater than or equal to 1 and is less than or equal to the N, and the N is positive integer；
In the embodiment of the present application, destination request data to be sent can be included by client according to data copy system
Node data carries out cutting, N number of section that the N number of data slot transmitted in parallel for then obtaining cutting includes to data copy system
Point.For i-th of node in N number of node, which can receive i-th of data slice in N number of data slot
Then section gives i-th of data slot transmitted in parallel to other nodes, and receives other data slices that other nodes are sent respectively
Section can complete answering for destination request data by storing to i-th of data slot and other data slots later
System.
By giving transmitted in parallel after data slot progress cutting to each node, each node receives corresponding data slice
It is mutually replicated again after section, data copying efficiency so can be improved, and then improve system performance, and since ask the visitor in for institute
The problem of asking the duplication of data without first passing through Leader node, therefore performance bottleneck being become to avoid Leader node.
Optionally, each data slot in N number of data slot carries fragment identification, and the fragment identification is described
It after client carries out cutting to the destination request data, adds and obtains in each data slot, the fragment identification is used
In position of the instruction corresponding data segment in the destination request data；
I-th of node stores i-th of data slot and other described data slots, comprising:
I-th of node acquisition request reception sequence, the request reception sequence are used to indicate data duplication system
The reception sequence for received all request datas of uniting, all request datas include the destination request data；
I-th of node is according to the request reception sequence and i-th of data slot and other described data
The fragment identification of segment stores i-th of data slot and other described data slots.
By acquisition request reception sequence, then according to request reception sequence and the fragment identification of each data slot,
Received data slot is stored, it is ensured that each data slot can sequentially form the destination request data, and can
Storage order to guarantee the request data in each node is consistent, and random ordering will not occur, so as to support multiple client
Data write-in, improve the scalability of the data copy system.
Optionally, each data slot in N number of data slot also carries the mark of the destination request data；
I-th of node acquisition request reception sequence, comprising:
When i-th of node is to be directed to the Leader node of the destination request data in N number of node, according to
The mark for the destination request data that i-th of data slot carries and the receiving time of i-th of data slot, with
And the mark of other received request datas and connecing for corresponding data segment in addition to the mark of the destination request data
Between time receiving, the request reception sequence is determined, and give the request reception sequence transmitted in parallel to other described nodes；
When i-th of node is to be directed to the Follower node of the destination request data in N number of node, connect
The request reception sequence that the Leader node in N number of node is sent is received, the request reception sequence is described
Leader node is according to the marks of the destination request data carried from the received data slot of the client and described connects
The receiving time of the data slot of receipts, and other received request datas in addition to the mark of the destination request data
Mark and corresponding data segment receiving time determination obtain.
By the mark by the request data based on the received of Follower node in N number of node, determine that the data replicate
The request reception sequence of system, and determining request is received into sequence synchronization to other Follower nodes, it is ensured that it is each
The sequence of the request data of node is consistent.
Optionally, i-th of node is according to request reception sequence and i-th of data slot and described
The fragment identification of other data slots stores i-th of data slot and other described data slots, comprising:
I-th of node determines the storage location of the destination request data according to the request reception sequence；
I-th of node determines institute according to the fragment identification of i-th of data slot and other data slots
State putting in order for i-th of data slot and other data slots；
I-th of node puts in order according to i-th of data slot and other data slots, will be described
I-th of data slot and other described data slots are stored in the storage location.
Optionally, each data slot in N number of data slot carries the mark of the destination request data；
I-th of node in N number of node receives i-th in N number of data slot of the client transmitted in parallel
After data slot, further includes:
According to the mark for the destination request data that i-th of data slot carries, the destination request number is determined
According to N number of index marker position, N number of index marker position and the N number of data slot correspond, N number of index marker
The initial mark position of position is the first flag bit, is used to indicate corresponding data segment and does not receive；
I-th of index marker position in N number of index marker position is changed to the second mark from first flag bit
Position, second flag bit are used to indicate corresponding data segment and have received；
After other data slots that other nodes described in the reception are sent respectively, further includes:
Other index marker positions are changed to second flag bit, other described index markers from first flag bit
Position refers to the index marker position in N number of index marker position in addition to i-th of index marker position；
I-th of node stores i-th of data slot and other described data slots, comprising:
When each index marker position in N number of index marker position is second flag bit, i-th of section
Point stores i-th of data slot and other described data slots.
Optionally, i-th of node carries out storing it to i-th of data slot and other described data slots
Afterwards, further includes:
I-th of node sends confirmation message to the client, and the confirmation message is used to indicate i-th of section
Point stores the destination request data and completes, and receives the confirmation message that N number of node is sent by the client
When, confirm that the destination request data replicate completion in the data copy system.
By sending confirmation message to client, each node replicate data of timely learning can be enabled a client to
Dynamically, and then in time determine whether destination request data replicate completion in the data copy system.
Second aspect provides a kind of data copy device, and described device, which has, realizes that data are multiple in above-mentioned first aspect
The function of method behavior processed.The data copy device includes at least one module, at least one module is for realizing above-mentioned
Data copy method provided by first aspect.
The third aspect provides a kind of node device, includes processor and memory, institute in the structure of the node device
Memory is stated for storing the program that support node apparatus executes data copy method provided by above-mentioned first aspect, Yi Jicun
Storage is for realizing data involved in data copy method provided by above-mentioned first aspect.The processor is configured to being used for
Execute the program stored in the memory.The operating device of the storage equipment can also include communication bus, and the communication is total
Line is for establishing connection between the processor and memory.
Fourth aspect provides a kind of computer readable storage medium, is stored in the computer readable storage medium
Instruction, when run on a computer, so that computer executes data copy method described in above-mentioned first aspect.
5th aspect, provides a kind of computer program product comprising instruction, when run on a computer, so that
Computer executes data copy method described in above-mentioned first aspect.
In above-mentioned second aspect, the third aspect, fourth aspect and the 5th aspect technical effect obtained and first aspect
The technical effect that corresponding technological means obtains is approximate, repeats no more herein.
Technical solution provided by the present application has the benefit that
In the embodiment of the present application, destination request data to be sent can be included by client according to data copy system
Node data carries out cutting, N number of section that the N number of data slot transmitted in parallel for then obtaining cutting includes to data copy system
Point.For i-th of node in N number of node, which can receive i-th of data slice in N number of data slot
Then section gives i-th of data slot transmitted in parallel to other nodes, and receives other data slices that other nodes are sent respectively
Section can complete answering for destination request data by storing to i-th of data slot and other data slots later
System.For the application by giving transmitted in parallel after data slot progress cutting to each node, each node receives corresponding data
It is mutually replicated again after segment, data copying efficiency so can be improved, and then improve system performance, and due to all
The duplication of request data is without first passing through Leader node, therefore the problem of can become performance bottleneck to avoid Leader node.
Detailed description of the invention
Figure 1A is a kind of structural schematic diagram of data copy system provided by the embodiments of the present application；
Figure 1B is a kind of logical construction schematic diagram of node 20 provided by the embodiments of the present application；
Fig. 1 C is a kind of structural schematic diagram of node device provided by the embodiments of the present application；
Fig. 1 D is a kind of flow chart of data copy method provided by the embodiments of the present application；
Fig. 1 E is a kind of schematic diagram of data duplication process provided by the embodiments of the present application；
Fig. 1 F is a kind of logical construction schematic diagram of i-th of node provided by the embodiments of the present application；
Fig. 1 G is a kind of flow diagram of the leader follower replication method provided in the related technology；
Fig. 1 H is a kind of flow diagram of data copy method provided by the embodiments of the present application；
Fig. 2 is a kind of structural schematic diagram of data copy device provided by the embodiments of the present application.
Specific embodiment
To keep the purposes, technical schemes and advantages of the application clearer, below in conjunction with attached drawing to the application embodiment party
Formula is described in further detail.
Before describing in detail to the embodiment of the present application, first the application scenarios of the embodiment of the present application are introduced.
The embodiment of the present invention is adapted in high reliability scene.In high reliability scene, in order to improve the reliable of business
Property and availability, it usually needs by business datum multiple nodes carry out redundancy backup, that is to say, in multiple nodes to business number
It, also can be by other node adapter tube business, to guarantee the reliable of data in this way when part of nodes delay machine according to being replicated
Property.Wherein, multiple nodes for carrying out redundancy backup to data are commonly known as copy.
In practical application, method provided by the embodiments of the present application be can be applied in the messaging service in public cloud.It is publicly-owned
Cloud refers to that third party provider is the shared resource service system that user provides.Messaging service refers to message intermediary service, is used for
Message is sent among applications or in distributed system.
Next the system architecture of the embodiment of the present application is introduced.
Figure 1A is a kind of structural schematic diagram of data copy system provided by the embodiments of the present application, as shown in Figure 1A, system
It including client 10 and N number of node 20, can be attached by network between client 10 and each node 20, each node
It can also be attached by network between 20.
Wherein, client 10 is used for the interstitial content N for including according to data copy system, to sent destination request number
According to cutting is carried out, N number of data slot is obtained, and give N number of data slot transmitted in parallel to N number of node 20.
Wherein, N number of node 20 for receiving N number of data slot of 10 transmitted in parallel of client, and each node 20 respectively
After receiving corresponding data slot, the data slot received can also mutually be replicated, so as to each node 20
This N number of data slot can be received, and this N number of data slot is stored, the destination request data are answered with realizing
System.
Specifically, for i-th of node 20 in N number of node 20, i-th of node 20 is for receiving client
I-th of data slot in N number of data slot of transmitted in parallel gives i-th of data slot transmitted in parallel to other nodes, and
Other data slots that other nodes are sent respectively are received, then i-th of data slot and other data slots are deposited
Storage, to complete the duplication of the destination request data.I is greater than or equal to 1 and is less than or equal to N.
It should be noted that the embodiment of the present application is said so that the data copy system includes a client as an example
It is bright, and in practical application, which can also include multiple client, and any client is equal in multiple client
Request data can be sent in the manner described above, and correspondingly, each node can in the manner described above send any client
Request data replicated.
In practical application, which can be physical node, or logical node, the embodiment of the present application to this not
It limits.For example, the node 20 can be terminal or server etc. when the node 20 is physical node；When the node is
When logical node, which can be virtual machine node or process node etc..
Further, which may include in operation system, be created in advance from business dubbing system
It builds to obtain.The operation system includes requests multiple for executing the node of business more than or equal to N, sending for client
Data can select N number of node as the copy of the request data, from multiple node to pass through N number of node to client
The request data that end is sent is replicated, and such client and N number of node also just constitute the data copy system.
For example, it is assumed that operation system includes 10 nodes, 3 nodes can be selected as number of request from this 10 nodes
According to copy, such client and this 3 nodes also just constitute the data copy system.
Figure 1B is a kind of logical construction schematic diagram of node 20 provided by the embodiments of the present application, as shown in Figure 1B, the node
20 may include Hash (Hash) cache module 21, sequence (Order) cache module 22 and log (Log) module 23.
Hash cache module 21 is used to save the data slot of the request data received.Further, which caches
Module 21 is also used to store the corresponding N number of index marker position of each request data, and N number of index marker position and the request data
N number of data slot correspond, be used to indicate whether corresponding data segment has received.
For example, the initial mark position of N number of index marker position can be the first flag bit, it is used to indicate corresponding data segment
It does not receive, when corresponding data segment receives, manipulative indexing flag bit can be changed to from the first initial flag bit
Second flag bit, the second flag bit are used to indicate corresponding data segment and have received.
For Order cache module 22 for recording request reception sequence, which is used to indicate data duplication
System receives the sequence of request data, and then is used to indicate the storage order to request data.
Log module 23 is for storing request data.It specifically, can be according to the cutting of request reception sequence and data slot
Sequentially, each data slot that request data includes is stored.
Next by combine Figure 1A to the invention relates to node 10 locating for the structure of node device be situated between
It continues.
Fig. 1 C is a kind of structural schematic diagram of node device provided by the embodiments of the present application, which can be terminal
Or the equipment such as server, referring to Fig. 1 C, the node device mainly include transmitter 110, receiver 120, memory 130,
Processor 140 and communication bus 150.It will be understood by those skilled in the art that the structure of node device shown in Fig. 1 C is simultaneously
Do not constitute the restriction to node device, in practical application, node device may include than illustrating more or fewer components, or
It combines certain components or different component layouts, the embodiment of the present application does not limit this.
Wherein, the transmitter 110 and receiver 120 are used to be communicated with other equipment, for example can pass through client
The data slot of transmission.The memory 130 can be used for storing data, for example can be used for storing data segment, also, this is deposited
Reservoir 130 can be used for storing one or more operation journeys for executing data copy method provided by the embodiments of the present application
Sequence and/or module.
Wherein, which is the control centre of node device, which can be at a general purpose central
It manages device (Central Processing Unit, CPU), microprocessor, application-specific integrated circuit (Application-
Specific Integrated Circuit, ASIC), or one or more held for controlling the embodiment of the present application scheme processes
Capable integrated circuit.The processor 140 can be by running or executing the software program being stored in memory 130 and/or mould
Block, and the data being stored in memory 130 are called, data copy method provided by Lai Shixian Examples below.
Wherein, which may include access, transmit information between above-mentioned processor 140 and memory 130.
Next a kind of data copy method provided by the embodiments of the present application will be described in detail.
Fig. 1 D is a kind of flow chart of data copy method provided by the embodiments of the present application, and this method can be applied to above-mentioned
In the data copy system that Figure 1A is somebody's turn to do, which includes client and N number of node, as shown in figure iD, this method packet
Include following steps:
Step 101: client carries out cutting according to N number of node, to sent destination request data, obtains N number of number
According to segment.
Wherein, which is the pre-set node for being replicated to the request data that the client is sent,
It that is to say, which is the copy that the pre-set request data for sending to the client carries out redundancy backup, institute
Stating N is pre-set copy number.The N is positive integer, and in practical application, N usually can be with value for 3 or 5 etc..
Wherein, client can obtain the information of N number of node from management node, and the management node is N number of for storing this
The information of node, the information of N number of node are used to indicate N number of node, specifically for indicating the number of N number of node and being somebody's turn to do
The position of N number of node.The management node can be any node in N number of node, or in addition to N number of node
Node, the embodiment of the present application are not construed as limiting this.
Specifically, which can receive this to the management node sending node information acquisition request, management node
After nodal information acquisition request, the information of N number of node of storage can be sent to the client.In practical application, visitor
Family end can also obtain the information of N number of node by metadata information.Metadata information is the data for describing data, mainly
The information for describing data attribute (property), is used to refer to the function such as storage location, historical data, resource lookup or file record
Energy.In the embodiment of the present application, which can store metadata information, which is used to indicate N number of node
Information, client can to the management node send metadata acquisition request, management node receive the metadata acquisition ask
After asking, the metadata information of storage can be sent to the client, so that by metadata information to obtain this N number of for client
The information of node.
Before sending destination request data to the data copy system, which can be first according to N number of node
Information determines the cutting number of request data, then cutting is carried out to the destination request data according to the cutting number, to obtain
N number of data slot.Specifically, client can averagely cut the destination request data according to the number of N number of node
Point, to obtain the identical N number of data slot of length.Alternatively, the client can also be according to the bandwidth of N number of node, to the mesh
It marks request data and carries out equal proportion cutting, obtain N number of data slot.
Further, which may include multiple message to be sent.It that is to say, which can incite somebody to action
Multiple message to be sent are assembled, and the destination request data are obtained, then according to according to the number of N number of node by the target
Number of request is cut into N parts.
Further, it during which carries out cutting to the destination request data, can be obtained in cutting every
The mark and fragment identification of the destination request data are added in a data slot.The mark of the destination request data is for unique
The destination request data are identified, can be title, number or the identity number of the destination request data
(Identification, ID) etc..The fragment identification is used to indicate the cutting sequence of corresponding data segment, and then can be used for referring to
Show position of the corresponding data segment in the destination request data.
For example, client, which carries out cutting to the destination request data, can successively obtain 3 data slots when N is 3,
And fragment identification 1 can be added in first data slot, fragment identification 2 is added in second data slot, in third
Fragment identification 3 is added in a data slot.
It is exemplary, referring to Fig. 1 E, it is assumed that the data copy system includes 3 nodes, respectively node 1, node 2 and node
3, client destination request data to be sent be Requst1, then can according to interstitial content 3 to Requst1 carry out cutting according to
Secondary to obtain 3 data slots, this 3 data slots are respectively R1P1, R1P2 and R1P3.Wherein, R1 is destination request data
Mark, is used to indicate Requst1.P1, P2 and P3 be fragment flag, be used to indicate corresponding data segment cutting sequence and
Position in the destination request data.
Step 102: client gives N number of data slot transmitted in parallel to N number of node.
After obtaining N number of data slot, which can be according to the rule of the corresponding node of a data slot, will
N number of data slot transmitted in parallel gives N number of node.Correspondingly, which can receive N number of data slot respectively, and
Each node is for receiving a data slot.
It, can be by R1P1, R1P2 and R1P3 transmitted in parallel to node 1, node 2 and node 3 for example, with reference to Fig. 1 E.
Step 103: i-th of node in N number of node receives i-th of data slot in N number of data slot.
For i-th of node in N number of node, which be can receive in N number of data slot
I-th of data slot.The i is greater than or equal to 1 and is less than or equal to the N.
For example, when N is 3, and when client obtains 3 data slots to the destination request data cutting, in this 3 nodes
First node can receive first data slot in this 3 data slots, second node can receive second
Data slot, third node can receive third data slot.
Step 104: i-th of node gives i-th of data slot transmitted in parallel to other nodes, and receives other nodes
Other data slots sent respectively.
Wherein, which refers to the node in N number of node in addition to i-th of node, other data slots
Refer to the data slot in N number of data slot in addition to i-th of data slot.
I-th of node by i-th of the data slot received while can be sent respectively to other nodes, so as to it
His node can be respectively received i-th of data slot, and correspondingly, i node can also receive its of other nodes transmission
His data slot.It that is to say, which can be sent to other data slots for the data slot received respectively, so as to
The data slot received is mutually replicated so that each node can receive that the destination request data include it is N number of
Data slot.
For example, with reference to Fig. 1 E, node 1 can be by the R1P1 transmitted in parallel received to node 2 and node 3, and node 2 can be with
By the R1P2 transmitted in parallel received to node 1 and node 2, the R1P3 transmitted in parallel received can be given node 1 He by node 3
Node 2.Correspondingly, the R1P3 that node 1 can distinguish the R1P2 of the transmission of receiving node 2 and node 3 is sent, node 2 can be distinguished
The R1P2 that the R1P1 and node 2 that receiving node 1 is sent are sent, node 3 can distinguish the R1P1 and node 2 of the transmission of receiving node 1
The R1P2 of transmission.
Further, from other nodes receive other nodes send data slot after, can also be to other
Node sends confirmation response, and confirmation response is used to indicate the data slot for having received transmission, to avoid the repetition of other nodes
Send data slot.
In the related technology, as interstitial content, that is, copy number increases, although system reliability increases, number of copies
Mesh is more, and the copy number that Leader node needs to wait is also more, and time delay increases, and system performance reduces, data duplication
Efficiency is lower.Moreover, because the duplication of all request datas requires to first pass through Leader node, therefore when data volume is more than
When the ability to bear of Leader node, Leader node will become the bottleneck of data duplication.
In the embodiment of the present application, cutting is carried out to destination request data according to interstitial content by client, cutting is obtained
The data slot transmitted in parallel arrived give N number of node, each node receive client transmission data slot after again mutually simultaneously
Row duplication realizes a kind of adaptive dynamic cutting streaming requests data of client and reproduction replica mechanism mostly living, so may be used
To effectively reduce the problem of causing time delay to increase because copy increases, data copying efficiency is improved.Moreover, each node can
To receive data slot and synchrodata segment simultaneously, the problem of single Leader node is as performance bottleneck is avoided.
Further, when each data slot in N number of data slot carries the mark of the destination request data,
It, can also be according to i-th of data slot after i-th of node receives i-th of data slot of the client transmitted in parallel
The mark of the destination request data carried, determines N number of index marker position of the destination request data, by N number of index marker
I-th of index marker position in position is changed to the second flag bit from first flag bit.Wherein, N number of index marker position and the N
A data slot corresponds, and the initial mark position of N number of index marker position is the first flag bit, is used to indicate corresponding data piece
Duan Wei is received, which is used to indicate corresponding data segment and has received.
Correspondingly, after receiving other data slots that other nodes are sent respectively, which can be with
Other index marker positions are changed to second flag bit from first flag bit, which refers to N number of rope
Draw the index marker position in flag bit in addition to i-th of index marker position.
It that is to say, it, can be by N number of index marker whenever receiving a data slot in the destination request data
Index marker corresponding with data slot position is changed to the second flag bit from the first original flag bit in position, to pass through change
Index marker position instruction corresponding data segment afterwards has received.
Wherein, first flag bit and the second flag bit can be preset.For example, first flag bit can be 0, the
Two flag bits can be 1.Assuming that first flag bit is 0, the second flag bit is 1, then when receiving i-th of data slot,
I-th of index marker position in N number of index marker position can be changed to 1 from 0, and can correspond to and save i-th of data
Segment.
Wherein, which can be arranged to obtain by i-th of node previously according to N number of node, can also be with
By i-th of node after receiving i-th of data slot, according to the destination request number of i-th of data slot carrying
According to mark and N number of node be arranged to obtain.
In one embodiment, which can be arranged a Hash Map previously according to N number of node, should
Hash Map includes the corresponding N number of index marker position of multiple request datas to be received, carries destination request data when receiving
After i-th of data slot of mark, N number of index marker can be chosen from multiple N number of index marker position as the mesh
Mark the corresponding N number of index marker position of request data.
In another embodiment, which can not also preset index marker position, but take receiving
I-th of data slot with destination request Data Identification and then N number of index marker is set for the destination request data.
For example, with reference to Fig. 1 E, it is assumed that client has continuously transmitted 2 destination request data, this 2 destination request data point
It Wei Requst1 (R1) and Requst2 (R2), wherein R1 includes data slot R1P1, R1P2 and R1P3, and R2 includes data slice
Section R2P1, R2P2 and R2P3, then referring to Fig. 1 F, i-th of node may include Hash cache module, Hash cache module storage
There are the corresponding index marker of R1 (IndexFlag) field and the corresponding IndexFlag field of R2.Wherein, R1 is corresponding
IndexFlag field include 3 index marker positions, this 3 index marker positions respectively with data slot P1, P2 and P3 couple of R1
It answers, is used to indicate whether corresponding data segment has received.The corresponding IndexFlag field of R2 also includes 3 index marker positions,
This 3 index marker positions are corresponding with data slot P1, P2 and P3 of R2 respectively, are used to indicate whether corresponding data segment has received
It arrives.
Step 105: i-th of node stores i-th of data slot and other data slots, to complete the mesh
Mark the duplication of request data.
Specifically, it may include following which, which carries out storage to i-th of data slot and other data slots,
Two kinds of implementations:
The first implementation: each data slot in N number of data slot carries fragment identification, i-th of node
According to the fragment identification of i-th of data slot and other data slots, to i-th of data slot and other data slices
Duan Jinhang storage.Wherein, which is after the client carries out cutting to the destination request data, in each data slice
It adds and obtains in section, which is used to indicate position of the corresponding data segment in the destination request data.
I-th of node can determine determination according to the fragment identification of i-th of data slot and other data slots
I-th of data slot puts in order with other data slots, then puts in order according to this to i-th of data slot
It is stored with other data slots, so that the N number of data slot finally stored can form the mesh according to storage order
Mark request data.
For example, with reference to Fig. 1 F, for data slot R1P1, R1P2 and the R1P3 received, can according to fragment identification P1,
P2 and P3 determines the putting in order as R1P1, R1P2 and R1P3 of this 3 data slots, then puts in order successively according to this to this
3 data slots are stored.It, can also be according to segment accordingly for data slot R2P1, R2P2 and the R2P3 received
Mark P1, P2 and P3 determine the putting in order as R2P1, R2P2 and R2P3 of this 3 data slots, then put in order according to this
Successively this 3 data slots are stored.
Second of implementation: each data slot in N number of data slot carries fragment identification, i-th of node
Acquisition request reception sequence, the request reception sequence are used to indicate connecing for the received all request datas of the data copy system
Receipts sequence, which includes the destination request data；I-th of node is according to the request reception sequence, and is somebody's turn to do
The fragment identification of i-th data slot and other data slots carries out i-th of data slot and other data slots
Storage.
Wherein, when each data slot in N number of data slot also carries the mark of the destination request data, this
I node can pass through following two mode acquisition request reception sequence:
1) when i-th of node is to be directed to the Leader node of the destination request data in N number of node, according to this
The mark for the destination request data that i data slot carries and the receiving time of i-th of data slot, and remove the target
The mark of other received request datas except the mark of request data and the receiving time of corresponding data segment, determining should
Reception sequence is requested, and gives the request reception sequence transmitted in parallel to other nodes.
Wherein, the Leader node in N number of node and Follower node can be preset according to duplication object
It arrives.When Leader node receives the data slot of new request data, it will store the number of request of data slot carrying
According to mark, then according to before the data slot for receiving the new request data mark of stored request data and
The mark of this new request data determines request reception sequence, and request reception sequence is sent to other Follower section
Point.
For example, with reference to Fig. 1 E, when i-th of node is to save in N number of node for the Leader of the destination request data
Point, and i-th of node first receives data slot R1P1, then when receiving data slot R2P1, then it can be according to number of request
According to mark R1 and R2 determine request reception sequence be R1-R2, request reception sequence R1-R2 be used to indicate the data duplication system
System first receives R1 and receives R2 again.Moreover, the Leader node can also give the request reception sequence R1-R2 transmitted in parallel
Node 2 and node 3.
Further, referring to Fig. 1 F, i-th of node can also include Order cache module, which deposits
The reception sequence of the request data of the data copy system is contained, reception sequence is R1-R2.
2) when i-th of node is to be directed to the Follower node of the destination request data in N number of node, receiving should
Request reception sequence that Leader node in N number of node is sent, request reception sequence be the Leader node according to from
The mark for the destination request data that the received data slot of the client carries and the receiving time of the received data slot,
And the mark of other received request datas and connecing for corresponding data segment in addition to the mark of the destination request data
Determination obtains between time receiving.
Specifically, i-th of node is according to the request reception sequence and i-th of data slot and other data
The fragment identification of segment, carrying out storage to i-th of data slot and other data slots may include: i-th of node
According to the request reception sequence, the storage location of the destination request data is determined；I-th of node is according to i-th of data slice
The fragment identification of section and other data slots determines putting in order for i-th of data slot and other data slots；It should
I-th of node puts in order according to i-th of data slot and other data slots, by i-th of data slot and should
Other data slots are stored in the storage location.
Wherein, which is used to indicate storage position of the destination request data in all received request datas
It sets.For example, with reference to Fig. 1 F, it is assumed that the destination request data are R1, then can determine depositing for R1 according to request reception sequence R1-R2
Storage space is set before the storage location of R2, correspondingly, each data slot R1P1, R1P2 of R1 can be successively stored before R2
And R1P3.
It is possible to further which i-th of data slot and other data slots are stored in the specified of i-th of node
In storage medium, which can be specified file or other storage mediums.Moreover, by i-th of data slice
Section and other data slots are stored in after the specified storage medium of i-th of node, are not limited to synchronous waiting rule
Success.
In the related technology, in order to reduce the dependence in data reproduction process to Leader node, it is also provided with one kind
Quorum replication strategy needs to write (Writer) process by one in Quorum replication strategy and is responsible for specially writing to each node
Enter request data, and needs that each node is waited just to indicate that request data replicates successfully when being all written successfully.But due to every
A node is all data to be written by the same Writer process, therefore the sequence between each node must be consistent,
It is that simultaneously data can only be written by a Writer process in the same node, just can guarantee the number of request of each node write-in
According to sequence consensus, if there is multiple Writer processes simultaneously toward each node write-in data when, due to each node reception when
Between it is inconsistent, will be unable to guarantee that each node receives the sequence of request data, to constrain the scalability of write operation.
And in the embodiment of the present application, by by Leader node determine request reception sequence, Leader node to
It can also be to Follower node by request reception sequence, so that each section while Follower node synchrodata segment
Point can receive sequential storage request data according to identical request, can so guarantee the request data of each node storage
Sequence consensus can support multiple client to send request data to N number of node simultaneously, and that is to say can support multiple clients
It holds while data is written, solve the scaling concern of write operation.
Further, i-th of node can also each index marker position in N number of index marker position be this
When two flag bits, i-th of data slot and other data slots are stored.
For example, with reference to Fig. 1 E and Fig. 1 F, which can be received by the request in traversal Order cache module
Sequentially, according to the corresponding IndexFlag field of request data each in request reception sequential search Hash cache module, determination pair
Answer whether each index marker position in IndexFlag field is all the second index marker position, when in corresponding IndexFlag field
Each index marker position when being all the second index marker position, Log module is written into each data slot of corresponding requests data.
Further, after storing to i-th of data slot and other data slots, i-th of node is also
Confirmation message can be sent to the client, which is used to indicate i-th of node and stores to the destination request data
It completes, when receiving the confirmation message that N number of node is sent by the client, confirms that the destination request data are multiple in the data
It replicates and completes in system processed.
Wherein, which is the response to the client, which can also carry the destination request data
Mark, for indicating that client i-th of node stores the destination request data by the marks of the destination request data
It completes.
Client can wait this after sending N number of data slot of the destination request data to N number of nodal parallel
The response of N number of node.When receiving the confirmation message that N number of node is sent, the destination request data can be confirmed in the number
It is completed according to being replicated in dubbing system.
Further, the client N number of data slot that the destination request data are sent to N number of nodal parallel it
Afterwards, when receiving the confirmation message that N number of node is sent in preset duration, can continue to send other to N number of node
Request data indicates response timeout, this is N number of when not receiving the confirmation message that N number of node is sent in the preset duration
Node, which replicates the destination request data, to fail, and client needs send N number of data to N number of nodal parallel again at this time
Segment.
In order to which the data copying efficiency to data copy method provided by the embodiments of the present application is illustrated, next by phase
The doubling time of the leader follower replication method and data copy method provided by the embodiments of the present application that provide in the technology of pass compares
Explanation.
Fig. 1 G is a kind of flow diagram of the leader follower replication method provided in the related technology, as shown in Figure 1 G, the principal and subordinate
Clone method includes following process:
1, client sends request data to Leader node.
2, partitioned file is written in received request data by Leader node.
3, Leader node sends request data to Follower.
4, partitioned file is written in received request data by Follower node
5, Follower node sends confirmation message to Leader node, and request data is written successfully in instruction.
6, Leader node sends confirmation message to client, and instruction replicates successfully request data.
If indicating the propagation delay time of the request data with TSemd, Twrite indicates the write-in time delay of the request data,
Tack indicates the propagation delay time of confirmation message, then time t1=TSemd used in step 1, t2=Twrite used in step 2, step 3
Time t3=TSemd used, time t4=Twrite used in step 4, time t5=Tack used in step 5, time used in step 6
T6=Tack.Therefore, the total time t=t1+t2+t3+t4+t5+t6=2TSemd+2Twrite+2Tack of leader follower replication method.
Fig. 1 H is a kind of flow diagram of data copy method provided by the embodiments of the present application, as shown in fig. 1H, the number
Include following process according to clone method:
1, client sends N number of data slot of request data to N number of nodal parallel.
2, each node gives the data slot transmitted in parallel received to other nodes.
3, each node stores the N number of data slot received.
4, each node sends confirmation message to client, is stored and is completed to request data with instruction.
Wherein, due to step 1 send be request data 1/N, time T1=1/N*TSemd used in step 1, together
Reason, T2=1/N*TSemd.In addition, T3=Twrite, T4=Tack.Therefore, data copy method provided by the embodiments of the present application
Total time T=T1+T2+T3+T4=2/N*TSemd+Twrite+Tack.
By above-mentioned comparative analysis it is found that the total time of leader follower replication method is data duplication side provided by the embodiments of the present application
2~3 times of the total time of method, so when data duplication can be effectively reduced in data copy method provided by the embodiments of the present application
Prolong, system performance is obviously improved, and the promotion of system performance increases with increasing for copy number.
In the embodiment of the present application, destination request data to be sent can be included by client according to data copy system
Node data carries out cutting, N number of section that the N number of data slot transmitted in parallel for then obtaining cutting includes to data copy system
Point.For i-th of node in N number of node, which can receive i-th of data slice in N number of data slot
Then section gives i-th of data slot transmitted in parallel to other nodes, and receives other data slices that other nodes are sent respectively
Section can complete answering for destination request data by storing to i-th of data slot and other data slots later
System.For the application by giving transmitted in parallel after data slot progress cutting to each node, each node receives corresponding data
It is mutually replicated again after segment, data copying efficiency so can be improved, and then improve system performance, and due to all
The duplication of request data is without first passing through Leader node, therefore the problem of can become performance bottleneck to avoid Leader node.
Fig. 2 is a kind of structural schematic diagram of data copy device provided by the embodiments of the present application, which can be data
I-th of node in N number of node that dubbing system includes, as shown in Fig. 2, the device includes: receiving module 201, sending module
202 and memory module 203.
Receiving module 201, for executing the step of step 103 described in above-mentioned Fig. 1 D embodiment executes；
Sending module 202, for executing the sending step of the execution of step 104 described in above-mentioned Fig. 1 D embodiment；
The receiving module 201 is also used to execute the receiving step of the execution of step 104 described in above-mentioned Fig. 1 D embodiment；
Memory module 203, for executing the step of step 105 described in above-mentioned Fig. 1 D embodiment executes.
Optionally, each data slot in N number of data slot carries fragment identification, which is the client
After carrying out cutting to the destination request data, adds and obtain in each data slot, which is used to indicate correspondence
Position of the data slot in the destination request data；
The memory module 203 is used to execute second of implementation in step 105 described in above-mentioned Fig. 1 D embodiment.
Optionally, which includes acquiring unit, and the acquiring unit is for executing described in above-mentioned Fig. 1 D embodiment
Step 105 include second of implementation in step 1) or 2).
Optionally, which includes storage unit, which is used for:
According to the request reception sequence, the storage location of the destination request data is determined；
According to the fragment identification of i-th of data slot and other data slots, determines i-th of data slot and be somebody's turn to do
Other data slots put in order；
According to putting in order for i-th of data slot and other data slots, by i-th of data slot and this its
His data slot is stored in the storage location.
Optionally, each data slot in N number of data slot carries the mark of the destination request data；
The device further include:
Determining module, the mark of the destination request data for being carried according to i-th of data slot, determines the target
N number of index marker position of request data, N number of index marker position and N number of data slot correspond, N number of index marker
The initial mark position of position is the first flag bit, is used to indicate corresponding data segment and does not receive；
First change module, for by i-th of index marker position in N number of index marker position from first flag bit more
It is changed to the second flag bit, which is used to indicate corresponding data segment and has received；
Second change module, for the receiving module receive other data slots that other nodes are sent respectively it
Afterwards, other index marker positions are changed to second flag bit from first flag bit, which refers to that this is N number of
Index marker position in index marker position in addition to i-th of index marker position；
The memory module 203, for being second flag bit when each index marker position in N number of index marker position
When, i-th of data slot and other data slots are stored.
Optionally, which is also used to:
After the memory module 203 stores i-th of data slot and other data slots, to the client
End sends confirmation message, which is used to indicate i-th of node and stores completion to the destination request data, by the client
When termination receives the confirmation message that N number of node is sent, confirm that the destination request data replicate in the data copy system
It completes.
In the embodiment of the present application, destination request data to be sent can be included by client according to data copy system
Node data carries out cutting, N number of section that the N number of data slot transmitted in parallel for then obtaining cutting includes to data copy system
Point.For i-th of node in N number of node, which can receive i-th of data slice in N number of data slot
Then section gives i-th of data slot transmitted in parallel to other nodes, and receives other data slices that other nodes are sent respectively
Section can complete answering for destination request data by storing to i-th of data slot and other data slots later
System.For the application by giving transmitted in parallel after data slot progress cutting to each node, each node receives corresponding data
It is mutually replicated again after segment, data copying efficiency so can be improved, and then improve system performance, and due to all
The duplication of request data is without first passing through Leader node, therefore the problem of can become performance bottleneck to avoid Leader node.
It should be understood that data provided by the above embodiment are replicated in when carrying out data duplication, only with above-mentioned each function
The division progress of module can according to need and for example, in practical application by above-mentioned function distribution by different function moulds
Block is completed, i.e., the internal structure of device is divided into different functional modules, to complete all or part of function described above
Energy.In addition, data copy device provided by the above embodiment and data copy method embodiment belong to same design, it is specific real
Existing process is detailed in embodiment of the method, and which is not described herein again.
In another embodiment, a kind of computer readable storage medium is additionally provided, in the computer readable storage medium
It is stored with instruction, when run on a computer, so that computer executes data duplication side described in above-mentioned Fig. 1 D embodiment
Method.
In another embodiment, a kind of computer program product comprising instruction is additionally provided, when it is transported on computers
When row, so that computer executes data copy method described in above-mentioned Fig. 1 D embodiment.
In the above-described embodiments, can come wholly or partly by software, hardware, firmware or its any combination real
It is existing.When implemented in software, it can entirely or partly realize in the form of a computer program product.The computer program
Product includes one or more computer instructions.It is all or part of when loading on computers and executing the computer instruction
Ground is generated according to process or function described in the embodiment of the present application.The computer can be general purpose computer, special purpose computer,
Computer network or other programmable devices.The computer instruction may be stored in a computer readable storage medium, or
Person is transmitted from a computer readable storage medium to another computer readable storage medium, for example, the computer instruction
Can from a web-site, computer, server or data center by it is wired (such as: coaxial cable, optical fiber, data use
Family line (Digital Subscriber Line, DSL)) or wireless (such as: infrared, wireless, microwave etc.) mode to another net
Website, computer, server or data center are transmitted.The computer readable storage medium can be computer can
Any usable medium of access either includes the data storage such as one or more usable mediums integrated server, data center
Equipment.The usable medium can be magnetic medium (such as: floppy disk, hard disk, tape), optical medium (such as: digital versatile disc
(Digital Versatile Disc, DVD)) or semiconductor medium (such as: solid state hard disk (Solid State Disk,
SSD)) etc..
Those of ordinary skill in the art will appreciate that realizing that all or part of the steps of above-described embodiment can pass through hardware
It completes, relevant hardware can also be instructed to complete by program, the program can store in a kind of computer-readable
In storage medium, storage medium mentioned above can be read-only memory, disk or CD etc..
The above is embodiment provided by the present application, all in spirit herein and original not to limit the application
Within then, any modification, equivalent replacement, improvement and so on be should be included within the scope of protection of this application.
Claims (14)
1. a kind of data copy method, which is characterized in that the method is applied to data copy system, the data copy system
Including client and N number of node, which comprises
I-th of node in N number of node receives i-th of data in N number of data slot of the client transmitted in parallel
Segment, N number of data slot, which is the client, carries out cutting to sent destination request data according to N number of node
It obtains, the i is greater than or equal to 1 and is less than or equal to the N, and the N is positive integer；
I-th of node gives i-th of data slot transmitted in parallel to other nodes, and receives other nodes difference
Other data slots sent, other described nodes refer to the node in N number of node in addition to i-th of node, institute
It states other data slots and refers to data slot in N number of data slot in addition to i-th of data slot；
I-th of node stores i-th of data slot and other described data slots, to complete the target
The duplication of request data.
2. the method as described in claim 1, which is characterized in that each data slot carrying sheets in N number of data slot
Segment identification, the fragment identification are after the client carries out cutting to the destination request data, in each data slot
Middle addition obtains, and the fragment identification is used to indicate position of the corresponding data segment in the destination request data；
I-th of node stores i-th of data slot and other described data slots, comprising:
I-th of node acquisition request reception sequence, the request reception sequence have been used to indicate the data copy system
The reception sequence of received all request datas, all request datas include the destination request data；
I-th of node is according to the request reception sequence and i-th of data slot and other described data slots
Fragment identification, i-th of data slot and other described data slots are stored.
3. method according to claim 2, which is characterized in that each data slot in N number of data slot also carries
The mark of the destination request data；
I-th of node acquisition request reception sequence, comprising:
When i-th of node is to be directed to the main side Leader node of the destination request data in N number of node, according to
The mark for the destination request data that i-th of data slot carries and the receiving time of i-th of data slot, with
And the mark of other received request datas and connecing for corresponding data segment in addition to the mark of the destination request data
Between time receiving, the request reception sequence is determined, and give the request reception sequence transmitted in parallel to other described nodes；
When i-th of node is to be directed to the standby host end Follower node of the destination request data in N number of node,
The request reception sequence that the Leader node in N number of node is sent is received, the request reception sequence is described
Leader node is according to the marks of the destination request data carried from the received data slot of the client and described connects
The receiving time of the data slot of receipts, and other received request datas in addition to the mark of the destination request data
Mark and corresponding data segment receiving time determination obtain.
4. method as claimed in claim 2 or claim 3, which is characterized in that i-th of node according to request reception sequence,
And the fragment identification of i-th of data slot and other data slots, to i-th of data slot and it is described its
He stores data slot, comprising:
I-th of node determines the storage location of the destination request data according to the request reception sequence；
I-th of node determines described according to the fragment identification of i-th of data slot and other data slots
I data slot and other data slots put in order；
I-th of node puts in order according to i-th of data slot and other data slots, by described i-th
A data slot and other described data slots are stored in the storage location.
5. the method as described in claim 1-4 is any, which is characterized in that each data slot in N number of data slot
Carry the mark of the destination request data；
I-th of node in N number of node receives i-th of data in N number of data slot of the client transmitted in parallel
After segment, further includes:
According to the mark for the destination request data that i-th of data slot carries, the N of the destination request data is determined
A index marker position, N number of index marker position and N number of data slot correspond, at the beginning of N number of index marker position
Beginning flag bit is the first flag bit, is used to indicate corresponding data segment and does not receive；
I-th of index marker position in N number of index marker position is changed to the second flag bit, institute from first flag bit
It states the second flag bit and is used to indicate corresponding data segment and have received；
After other data slots that other nodes described in the reception are sent respectively, further includes:
Other index marker positions are changed to second flag bit from first flag bit, other described index marker positions are
Refer to the index marker position in N number of index marker position in addition to i-th of index marker position；
I-th of node stores i-th of data slot and other described data slots, comprising:
When each index marker position in N number of index marker position is second flag bit, i-th of node pair
I-th of data slot and other described data slots are stored.
6. method a method as claimed in any one of claims 1 to 5, which is characterized in that i-th of node is to i-th of data slot
After being stored with other described data slots, further includes:
I-th of node sends confirmation message to the client, and the confirmation message is used to indicate i-th of node pair
The destination request data storage is completed, when receiving the confirmation message that N number of node is sent by the client, really
Recognize the destination request data and replicates completion in the data copy system.
7. a kind of data copy device, which is characterized in that described device is applied in N number of node that data copy system includes
I-th of node, the data copy system include client and N number of node, and described device includes:
Receiving module, i-th of data slot in N number of data slot for receiving the client transmitted in parallel are described N number of
Data slot is that the client is obtained according to N number of node to sent the progress cutting of destination request data, and the i is big
In or be equal to 1 and be less than or equal to the N, the N be positive integer；
Sending module, for giving i-th of data slot transmitted in parallel to other nodes, other described nodes refer to the N
Node in a node in addition to i-th of node；
The receiving module is also used to receive other data slots that other described nodes are sent respectively, other described data slices
Section refers to the data slot in N number of data slot in addition to i-th of data slot；
Memory module, for being stored to i-th of data slot and other described data slots, to complete the target
The duplication of request data.
8. device as claimed in claim 7, which is characterized in that each data slot carrying sheets in N number of data slot
Segment identification, the fragment identification are after the client carries out cutting to the destination request data, in each data slot
Middle addition obtains, and the fragment identification is used to indicate position of the corresponding data segment in the destination request data；
The memory module, comprising:
Acquiring unit, for acquisition request reception sequence, the request reception sequence has been used to indicate the data copy system
The reception sequence of received all request datas, all request datas include the destination request data；
Storage unit, for according to the request reception sequence and i-th of data slot and other described data slots
Fragment identification, i-th of data slot and other described data slots are stored.
9. device as claimed in claim 8, which is characterized in that each data slot in N number of data slot also carries
The mark of the destination request data；
The acquiring unit is specifically used for:
When i-th of node is to be directed to the main side Leader node of the destination request data in N number of node, according to
The mark for the destination request data that i-th of data slot carries and the receiving time of i-th of data slot, with
And the mark of other received request datas and connecing for corresponding data segment in addition to the mark of the destination request data
Between time receiving, the request reception sequence is determined, and give the request reception sequence transmitted in parallel to other described nodes；
When i-th of node is to be directed to the standby host end Follower node of the destination request data in N number of node,
The request reception sequence that the Leader node in N number of node is sent is received, the request reception sequence is described
Leader node is according to the marks of the destination request data carried from the received data slot of the client and described connects
The receiving time of the data slot of receipts, and other received request datas in addition to the mark of the destination request data
Mark and corresponding data segment receiving time determination obtain.
10. device as claimed in claim 8 or 9, which is characterized in that the storage unit is specifically used for:
According to the request reception sequence, the storage location of the destination request data is determined；
According to the fragment identification of i-th of data slot and other data slots, determine i-th of data slot and
Other data slots put in order；
According to putting in order for i-th of data slot and other data slots, by i-th of data slot and institute
It states other data slots and is stored in the storage location.
11. the device as described in claim 7-10 is any, which is characterized in that each data slice in N number of data slot
Section carries the mark of the destination request data；
Described device further include:
Determining module, the mark of the destination request data for being carried according to i-th of data slot, determines the mesh
N number of index marker position of request data is marked, N number of index marker position and N number of data slot correspond, described N number of
The initial mark position of index marker position is the first flag bit, is used to indicate corresponding data segment and does not receive；
First change module, for by i-th of index marker position in N number of index marker position from first flag bit more
It is changed to the second flag bit, second flag bit is used to indicate corresponding data segment and has received；
Second change module, for the receiving module receive other data slots that other described nodes are sent respectively it
Afterwards, other index marker positions are changed to second flag bit from first flag bit, other described index marker positions are
Refer to the index marker position in N number of index marker position in addition to i-th of index marker position；
The memory module, for being second flag bit when each index marker position in N number of index marker position
When, i-th of data slot and other described data slots are stored.
12. the device as described in claim 7-11 is any, which is characterized in that the sending module is also used to:
After the memory module stores i-th of data slot and other described data slots, to the visitor
Family end sends confirmation message, and the confirmation message is used to indicate i-th of node and stores completion to the destination request data,
When receiving the confirmation message that N number of node is sent by the client, confirm the destination request data described
It replicates and completes in data copy system.
13. a kind of node device, the node device includes memory, processor and is stored on the memory and can be in institute
State the computer program run on processor, which is characterized in that the processor is configured to perform claim requires described in 1-6
The step of any one method.
14. a kind of computer readable storage medium, instruction is stored in the computer readable storage medium, which is characterized in that
When run on a computer, so that computer executes as the method according to claim 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711482591.0A CN109992447B (en) | 2017-12-29 | 2017-12-29 | Data copying method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711482591.0A CN109992447B (en) | 2017-12-29 | 2017-12-29 | Data copying method, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109992447A true CN109992447A (en) | 2019-07-09 |
| CN109992447B CN109992447B (en) | 2021-06-15 |
Family
ID=67109234
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711482591.0A Active CN109992447B (en) | 2017-12-29 | 2017-12-29 | Data copying method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109992447B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110868247A (en) * | 2019-11-20 | 2020-03-06 | 哈尔滨工业大学 | Event-driven satellite telemetry data processing method and device and computer storage medium |
| CN111182067A (en) * | 2019-12-31 | 2020-05-19 | 上海焜耀网络科技有限公司 | Data writing method and device based on interplanetary file system IPFS |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101046952A (en) * | 2006-03-29 | 2007-10-03 | 雅马哈株式会社 | Audio network system |
| CN101217678A (en) * | 2007-12-28 | 2008-07-09 | 华为技术有限公司 | System upgrade method, system and device |
| CN101419600A (en) * | 2007-10-22 | 2009-04-29 | 深圳市亚贝电气技术有限公司 | Data copy mapping method and device based on object-oriented LANGUAGE |
| CN101534205A (en) * | 2008-03-11 | 2009-09-16 | 中国网通集团宽带业务应用国家工程实验室有限公司 | Application layer multicast service realizing method, terminal and system thereof |
| CN103873173A (en) * | 2014-03-27 | 2014-06-18 | 北京悦迪文化传媒有限公司 | Digital pushing radio (DPR) method based on wireless digital radio technology |
| CN105406999A (en) * | 2015-11-20 | 2016-03-16 | 福建星网锐捷安防科技有限公司 | Multi-data distribution method and distribution equipment |
-
2017
- 2017-12-29 CN CN201711482591.0A patent/CN109992447B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101046952A (en) * | 2006-03-29 | 2007-10-03 | 雅马哈株式会社 | Audio network system |
| CN101419600A (en) * | 2007-10-22 | 2009-04-29 | 深圳市亚贝电气技术有限公司 | Data copy mapping method and device based on object-oriented LANGUAGE |
| CN101217678A (en) * | 2007-12-28 | 2008-07-09 | 华为技术有限公司 | System upgrade method, system and device |
| CN101534205A (en) * | 2008-03-11 | 2009-09-16 | 中国网通集团宽带业务应用国家工程实验室有限公司 | Application layer multicast service realizing method, terminal and system thereof |
| CN103873173A (en) * | 2014-03-27 | 2014-06-18 | 北京悦迪文化传媒有限公司 | Digital pushing radio (DPR) method based on wireless digital radio technology |
| CN105406999A (en) * | 2015-11-20 | 2016-03-16 | 福建星网锐捷安防科技有限公司 | Multi-data distribution method and distribution equipment |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110868247A (en) * | 2019-11-20 | 2020-03-06 | 哈尔滨工业大学 | Event-driven satellite telemetry data processing method and device and computer storage medium |
| CN110868247B (en) * | 2019-11-20 | 2022-02-25 | 哈尔滨工业大学 | Event-driven satellite telemetry data processing method and device and computer storage medium |
| CN111182067A (en) * | 2019-12-31 | 2020-05-19 | 上海焜耀网络科技有限公司 | Data writing method and device based on interplanetary file system IPFS |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109992447B (en) | 2021-06-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20200012568A1 (en) | Scalable log-based continuous data protection for distributed databases | |
| CN103268318B (en) | A kind of distributed key value database system of strong consistency and reading/writing method thereof | |
| JP4696089B2 (en) | Distributed storage system | |
| US9170892B2 (en) | Server failure recovery | |
| KR101905198B1 (en) | Distributed data storage | |
| CN102708165B (en) | Document handling method in distributed file system and device | |
| US8438134B2 (en) | Computer-readable recording medium containing database copying program, and database copying apparatus and method | |
| US9952940B2 (en) | Method of operating a shared nothing cluster system | |
| CN111182067B (en) | Data writing method and device based on interplanetary file system IPFS | |
| US9547706B2 (en) | Using colocation hints to facilitate accessing a distributed data storage system | |
| US8135918B1 (en) | Data de-duplication for iSCSI | |
| US20070078911A1 (en) | Replicating data across the nodes in a cluster environment | |
| CN113168404B (en) | System and method for replicating data in a distributed database system | |
| CN103885895A (en) | Write Performance in Fault-Tolerant Clustered Storage Systems | |
| CN106126374B (en) | Method for writing data, method for reading data and device | |
| US20190347167A1 (en) | Primary Node-Standby Node Data Transmission Method, Control Node, and Database System | |
| CN110099084B (en) | Method, system and computer readable medium for ensuring storage service availability | |
| US8447825B2 (en) | File sharing system and file sharing method | |
| CN106897288B (en) | Service providing method and system for database | |
| JP4208506B2 (en) | High-performance storage device access environment | |
| WO2016101662A1 (en) | Data processing method and relevant server | |
| CN109992447A (en) | Data copy method, device and storage medium | |
| CN107493309B (en) | File writing method and device in distributed system | |
| US10545667B1 (en) | Dynamic data partitioning for stateless request routing | |
| US9607068B2 (en) | Parallel truncation point management for log based replication |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200424 Address after: 518129 Bantian HUAWEI headquarters office building, Longgang District, Guangdong, Shenzhen Applicant after: HUAWEI TECHNOLOGIES Co.,Ltd. Address before: 301, A building, room 3, building 301, foreshore Road, No. 310052, Binjiang District, Zhejiang, Hangzhou Applicant before: Huawei Technologies Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |