CN109992447B - Data copying method, device and storage medium - Google Patents
Data copying method, device and storage medium Download PDFInfo
- Publication number
- CN109992447B CN109992447B CN201711482591.0A CN201711482591A CN109992447B CN 109992447 B CN109992447 B CN 109992447B CN 201711482591 A CN201711482591 A CN 201711482591A CN 109992447 B CN109992447 B CN 109992447B
- Authority
- CN
- China
- Prior art keywords
- data
- ith
- segment
- nodes
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1464—Management of the backup or restore process for networked environments
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Computer And Data Communications (AREA)
Abstract
The application discloses a data copying method, a data copying device and a storage medium, and belongs to the field of data processing. The method is applied to a data replication system, the data replication system comprises a client and N nodes, and the method comprises the following steps: the ith node in the N nodes receives the ith data segment in the N data segments which are sent by the client side in parallel, wherein the N data segments are obtained by the client side by segmenting target request data to be sent according to the N nodes; the ith data segment is sent to other nodes in parallel, and other data segments sent by the other nodes respectively are received; and storing the ith data segment and the other data segments to finish the copying of the target request data. The method and the device can improve the data copying efficiency and the system performance, and can avoid the problem that the Leader node becomes a performance bottleneck.
Description
Technical Field
The present application relates to the field of data processing, and in particular, to a data replication method, apparatus, and storage medium.
Background
Data replication is a data redundancy backup method, which can ensure the reliability and availability of a system by backing up data at a plurality of nodes in the service operation process, and switching the service of a node to other nodes when a certain node fails to take over the service through other nodes. The data replication method is particularly applied to a data replication system, and the data replication system comprises a client and a plurality of nodes, wherein the client is used for sending request data, and the nodes are used for replicating the request data sent by the client.
Currently, the most commonly used data replication method is a master-slave replication method, in which, for a plurality of nodes included in a data replication system, a certain node of the plurality of nodes may be set as a master (Leader) node in advance, and other nodes may be set as backup (Follower) nodes. In the data copying process, the client may first send the request data to the Leader node, so that the Leader node writes the received request data into the partition file. Each Follower node can pull the request data from the Leader node after the Leader node writes the request data into the partition file, and writes the request data into the partition file of the Follower node, and after the write is successful, a write success response can be sent to the Leader node. And after the Leader node receives the write success responses sent by all the Follower nodes in the plurality of nodes, the request data can be confirmed to be successfully copied.
In the master-slave replication method, because the request data needs to be replicated by the Leader node first and then the replication of the request data by each Follower node is waited, the replication efficiency is low and the performance is poor. Moreover, since all the requested data needs to be copied through the Leader node, the Leader node becomes a bottleneck of data copying when the data volume exceeds the bearing capacity of the Leader node.
Disclosure of Invention
The application provides a data replication method, a data replication device and a storage medium, which can be used for solving the problems of low replication efficiency, poor system performance and bottleneck of a Leader node in the related technology. The technical scheme is as follows:
in a first aspect, a data replication method is provided, where the method is applied to a data replication system, where the data replication system includes a client and N nodes, and the method includes:
receiving, by an ith node of the N nodes, an ith data segment of N data segments sent by the client side in parallel, where the N data segments are obtained by the client side by segmenting target request data to be sent according to the N nodes, where i is greater than or equal to 1 and less than or equal to N, and N is a positive integer;
in the embodiment of the application, the client can segment target request data to be sent according to node data included in the data replication system, and then send N data segments obtained by segmentation to N nodes included in the data replication system in parallel. For the ith node of the N nodes, the ith node may receive the ith data fragment of the N data fragments, then send the ith data fragment to other nodes in parallel, receive other data fragments respectively sent by other nodes, and then may complete the replication of the target request data by storing the ith data fragment and other data fragments.
The data segments are segmented and then are sent to the nodes in parallel, and the nodes copy the corresponding data segments after receiving the corresponding data segments, so that the data copying efficiency can be improved, the system performance is improved, and the problem that the Leader nodes become performance bottlenecks can be avoided because all the requested data do not need to be copied through the Leader nodes.
Optionally, each data fragment of the N data fragments carries a fragment identifier, where the fragment identifier is obtained by adding the fragment identifier to each data fragment after the client splits the target request data, and the fragment identifier is used to indicate a position of a corresponding data fragment in the target request data;
the ith node storing the ith data segment and the other data segments, including:
the ith node acquires a request receiving sequence, wherein the request receiving sequence is used for indicating a receiving sequence of all request data received by the data replication system, and the all request data comprise the target request data;
and the ith node stores the ith data segment and the other data segments according to the request receiving sequence and the segment identifications of the ith data segment and the other data segments.
By acquiring the request receiving sequence and storing the received data segments according to the request receiving sequence and the segment identifiers of the data segments, the target request data can be formed by the data segments in sequence, the storage sequence of the request data in each node can be consistent, disorder cannot occur, data writing of a plurality of clients can be supported, and the expansibility of the data replication system is improved.
Optionally, each data fragment of the N data fragments further carries an identifier of the target request data;
the obtaining, by the ith node, a request receiving order includes:
when the ith node is a Leader node aiming at the target request data in the N nodes, determining a request receiving sequence according to the identification of the target request data carried by the ith data segment and the receiving time of the ith data segment, as well as the identifications of other received request data except the identification of the target request data and the receiving time of the corresponding data segment, and sending the request receiving sequence to other nodes in parallel;
when the ith node is a fowler node of the N nodes for the target request data, receiving the request receiving sequence sent by a Leader node of the N nodes, where the request receiving sequence is determined by the Leader node according to the identifier of the target request data and the receiving time of the received data fragment carried by the data fragment received from the client, and the identifiers of other received request data except the identifier of the target request data and the receiving time of the corresponding data fragment.
The order of the request data of each node can be ensured to be consistent by determining the request receiving order of the data replication system by the following Follower nodes in the N nodes according to the identification of the received request data and synchronizing the determined request receiving order to other Follower nodes.
Optionally, the storing, by the ith node, the ith data segment and the other data segments according to the request receiving order and the segment identifiers of the ith data segment and the other data segments includes:
the ith node determines the storage position of the target request data according to the request receiving sequence;
the ith node determines the arrangement sequence of the ith data segment and the other data segments according to the segment identifiers of the ith data segment and the other data segments;
and the ith node stores the ith data segment and the other data segments in the storage position according to the arrangement sequence of the ith data segment and the other data segments.
Optionally, each data fragment of the N data fragments carries an identifier of the target request data;
after the ith node in the N nodes receives the ith data segment in the N data segments sent by the client side in parallel, the method further includes:
determining N index zone bits of the target request data according to the identifier of the target request data carried by the ith data segment, wherein the N index zone bits correspond to the N data segments one by one, and the initial zone bit of the N index zone bits is a first zone bit and is used for indicating that the corresponding data segment is not received;
changing an ith index flag bit of the N index flag bits from the first flag bit to a second flag bit, the second flag bit being used to indicate that a corresponding data segment has been received;
after receiving the other data segments respectively sent by the other nodes, the method further includes:
changing other index flag bits from the first flag bit to the second flag bit, the other index flag bits being index flag bits of the N index flag bits except for the ith index flag bit;
the ith node storing the ith data segment and the other data segments, including:
when each index flag bit of the N index flag bits is the second flag bit, the ith node stores the ith data segment and the other data segments.
Optionally, after the ith node stores the ith data segment and the other data segments, the method further includes:
and the ith node sends a confirmation message to the client, wherein the confirmation message is used for indicating that the ith node finishes storing the target request data, and when the client receives the confirmation messages sent by the N nodes, the client confirms that the target request data is completely copied in the data copying system.
By sending the confirmation message to the client, the client can timely know the dynamic state of the data copied by each node, and further timely determine whether the target request data is copied in the data copying system.
In a second aspect, there is provided a data copying apparatus having a function of implementing the behavior of the data copying method in the first aspect. The data replication device comprises at least one module, and the at least one module is used for implementing the data replication method provided by the first aspect.
In a third aspect, a node device is provided, where the structure of the node device includes a processor and a memory, and the memory is used for storing a program for supporting the node device to execute the data replication method provided in the first aspect, and storing data used for implementing the data replication method provided in the first aspect. The processor is configured to execute programs stored in the memory. The operating means of the memory device may further comprise a communication bus for establishing a connection between the processor and the memory.
In a fourth aspect, a computer-readable storage medium is provided, having stored therein instructions, which, when run on a computer, cause the computer to perform the data replication method of the first aspect described above.
In a fifth aspect, there is provided a computer program product containing instructions which, when run on a computer, cause the computer to perform the data replication method of the first aspect described above.
The technical effects obtained by the above second, third, fourth and fifth aspects are similar to the technical effects obtained by the corresponding technical means in the first aspect, and are not described herein again.
The beneficial effect that technical scheme that this application provided brought is:
in the embodiment of the application, the client can segment target request data to be sent according to node data included in the data replication system, and then send N data segments obtained by segmentation to N nodes included in the data replication system in parallel. For the ith node of the N nodes, the ith node may receive the ith data fragment of the N data fragments, then send the ith data fragment to other nodes in parallel, receive other data fragments respectively sent by other nodes, and then may complete the replication of the target request data by storing the ith data fragment and other data fragments. The data segment is divided and then parallelly sent to each node, each node receives the corresponding data segment and then copies the data segment, so that the data copying efficiency can be improved, the system performance is improved, and moreover, because the copying of all request data does not need to pass through a Leader node first, the problem that the Leader node becomes a performance bottleneck can be avoided.
Drawings
Fig. 1A is a schematic structural diagram of a data replication system according to an embodiment of the present application;
fig. 1B is a schematic logical structure diagram of a node 20 according to an embodiment of the present disclosure;
fig. 1C is a schematic structural diagram of a node device according to an embodiment of the present application;
fig. 1D is a flowchart of a data replication method provided in an embodiment of the present application;
fig. 1E is a schematic diagram of a data replication process provided in an embodiment of the present application;
fig. 1F is a schematic logical structure diagram of an ith node according to an embodiment of the present application;
FIG. 1G is a schematic flow chart of a master-slave replication method provided in the related art;
fig. 1H is a schematic flowchart of a data replication method provided in an embodiment of the present application;
fig. 2 is a schematic structural diagram of a data copying apparatus according to an embodiment of the present application.
Detailed Description
To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
Before describing the embodiments of the present application in detail, an application scenario of the embodiments of the present application is described first.
The embodiment of the invention is suitable for high-reliability scenes. In a high-reliability scenario, in order to improve reliability and availability of a service, it is usually necessary to perform redundant backup on service data at multiple nodes, that is, to copy the service data at multiple nodes, so that when some nodes are down, services can be taken over by other nodes, thereby ensuring reliability of the data. Wherein the plurality of nodes used to redundantly backup data are often referred to as replicas.
In practical application, the method provided by the embodiment of the application can be applied to message services on public clouds. The public cloud refers to a shared resource service system provided by a third-party provider for a user. A messaging service refers to a messaging intermediary service for sending messages between applications or in a distributed system.
Next, a system architecture of the embodiment of the present application is described.
Fig. 1A is a schematic structural diagram of a data replication system according to an embodiment of the present application, and as shown in fig. 1A, the system includes a client 10 and N nodes 20, where the client 10 and each node 20 may be connected through a network, and each node 20 may also be connected through a network.
The client 10 is configured to segment target request data to be sent according to the number N of nodes included in the data replication system to obtain N data segments, and send the N data segments to the N nodes 20 in parallel.
The N nodes 20 are configured to receive N data segments sent by the client 10 in parallel, and after each node 20 receives a corresponding data segment, the received data segments may be mutually copied, so that each node 20 can receive the N data segments, and store the N data segments, so as to copy the target request data.
Specifically, for the ith node 20 in the N nodes 20, the ith node 20 is configured to receive the ith data segment in the N data segments sent by the client in parallel, send the ith data segment to other nodes in parallel, receive other data segments sent by other nodes respectively, and then store the ith data segment and other data segments to complete the replication of the target request data. i is greater than or equal to 1 and less than or equal to N.
It should be noted that, in the embodiment of the present application, the data replication system is described by taking an example that the data replication system includes one client, and in practical applications, the data replication system may further include a plurality of clients, any client of the plurality of clients may send request data according to the foregoing manner, and accordingly, each node may replicate the request data sent by any client according to the foregoing manner.
In practical applications, the node 20 may be a physical node or a logical node, which is not limited in this embodiment of the present application. For example, when the node 20 is a physical node, the node 20 may be a terminal or a server, etc.; when the node is a logical node, the node 20 may be a virtual machine node or a process node, etc.
Further, the data replication system may be included in the business system and created in advance from the business replication system. The business system comprises a plurality of nodes which are larger than or equal to N and used for executing business, for request data sent by a client, N nodes can be selected from the plurality of nodes to be used as copies of the request data, so that the request data sent by the client is copied through the N nodes, and thus the client and the N nodes form the data copying system.
For example, assuming that the service system includes 10 nodes, 3 nodes can be selected from the 10 nodes as the copy of the requested data, so that the client and the 3 nodes also form the data replication system.
Fig. 1B is a schematic logical structure diagram of a node 20 according to an embodiment of the present disclosure, and as shown in fig. 1B, the node 20 may include a Hash (Hash) cache module 21, a sequential (Order) cache module 22, and a Log (Log) module 23.
The Hash cache module 21 is configured to store the data fragment of the received request data. Further, the HASH cache module 21 is further configured to store N index flag bits corresponding to each request data, and the N index flag bits are in one-to-one correspondence with N data segments of the request data, and are used to indicate whether corresponding data segments have been received.
For example, an initial flag bit of the N index flag bits may be a first flag bit for indicating that the corresponding data segment is not received, and when the corresponding data segment is received, the corresponding index flag bit may be changed from the initial first flag bit to a second flag bit for indicating that the corresponding data segment is received.
The Order cache module 22 is configured to record a request receiving sequence, where the request receiving sequence is used to indicate an Order in which the data replication system receives the requested data, and further is used to indicate a storage sequence of the requested data.
The Log module 23 is used to store the requested data. Specifically, each data fragment included in the request data may be stored according to the request receiving order and the slicing order of the data fragments.
Next, with reference to fig. 1A, a structure of a node device in which the node 10 according to an embodiment of the present application is located will be described.
Fig. 1C is a schematic structural diagram of a node device according to an embodiment of the present disclosure, where the node device may be a terminal or a server, and referring to fig. 1C, the node device mainly includes a transmitter 110, a receiver 120, a memory 130, a processor 140, and a communication bus 150. Those skilled in the art will understand that the structure of the node device shown in fig. 1C does not constitute a limitation to the node device, and in practical applications, the node device may include more or less components than those shown in the drawings, or combine some components, or arrange different components, which is not limited in the embodiments of the present application.
Wherein the transmitter 110 and the receiver 120 are used for communicating with other devices, such as data segments that can be sent by a client. The memory 130 may be used for storing data, such as data segments, and the memory 130 may also be used for storing one or more running programs and/or modules for executing the data copying method provided by the embodiment of the present application.
The processor 140 is a control center of the node device, and the processor 140 may be a general-purpose Central Processing Unit (CPU), a microprocessor, an Application-Specific Integrated Circuit (ASIC), or one or more Integrated circuits for controlling the execution of programs according to the embodiments of the present Application. The processor 140 may implement the data replication method provided by the embodiments below by running or executing software programs and/or modules stored in the memory 130 and calling data stored in the memory 130.
The communication bus 150 may include a path that transfers information between the processor 140 and the memory 130.
Next, a data copying method provided in an embodiment of the present application will be described in detail.
Fig. 1D is a flowchart of a data replication method according to an embodiment of the present application, where the method may be applied to the data replication system shown in fig. 1A, where the data replication system includes a client and N nodes, and as shown in fig. 1D, the method includes the following steps:
step 101: and the client divides the target request data to be sent according to the N nodes to obtain N data segments.
The N nodes are preset nodes for copying the request data sent by the client, that is, the N nodes are preset copies for performing redundant backup on the request data sent by the client, and N is a preset number of copies. N is a positive integer, and in practical application, N may generally take a value of 3 or 5, and the like.
The client may obtain information of the N nodes from a management node, where the management node is configured to store the information of the N nodes, and the information of the N nodes is used to indicate the N nodes, and specifically is used to indicate the number of the N nodes and positions of the N nodes. The management node may be any node in the N nodes, or may be a node other than the N nodes, which is not limited in this embodiment of the present application.
Specifically, the client may send a node information obtaining request to the management node, and after receiving the node information obtaining request, the management node may send the stored information of the N nodes to the client. In practical application, the client may further obtain information of the N nodes through metadata information. The metadata information is data describing data, mainly data attribute (property), and is used to indicate functions such as storage location, history data, resource search, file record, and the like. In this embodiment, the management node may store metadata information, where the metadata information is used to indicate information of the N nodes, the client may send a metadata obtaining request to the management node, and after receiving the metadata obtaining request, the management node may send the stored metadata information to the client, so that the client obtains the information of the N nodes through the metadata information.
Before sending target request data to the data replication system, the client may determine the number of segments of the request data according to the information of the N nodes, and then segment the target request data according to the number of segments to obtain N data segments. Specifically, the client may averagely segment the target request data according to the number of the N nodes to obtain N data segments with the same length. Or, the client may also perform equal-proportion segmentation on the target request data according to the bandwidths of the N nodes to obtain the N data segments.
Further, the target request data may include a plurality of messages to be transmitted. That is, the client may assemble a plurality of messages to be sent to obtain the target request data, and then divide the target request number into N according to the number of the N nodes.
Further, in the process that the client splits the target request data, the identifier of the target request data and the segment identifier may be added to each data segment obtained by splitting. The identifier of the target request data is used to uniquely identify the target request data, and may be a name, a number, an Identification number (ID), or the like of the target request data. The segment identifier is used to indicate the slicing order of the corresponding data segment, and thus may be used to indicate the position of the corresponding data segment in the target request data.
For example, when N is 3, the client splits the target request data to sequentially obtain 3 data fragments, and may add a fragment identifier 1 to the first data fragment, add a fragment identifier 2 to the second data fragment, and add a fragment identifier 3 to the third data fragment.
For example, referring to fig. 1E, assuming that the data replication system includes 3 nodes, which are node 1, node 2, and node 3, respectively, and target request data to be sent by the client is Requst1, Requst1 may be segmented according to the number of nodes 3 to obtain 3 data segments, where the 3 data segments are R1P1, R1P2, and R1P3, respectively. Where R1 is the identification of the target request data, indicating Requst 1. P1, P2, and P3 are fragment flags indicating the slicing order of the corresponding data fragments and the location in the target request data.
Step 102: and the client side sends the N data segments to the N nodes in parallel.
After obtaining N data segments, the client may send the N data segments to the N nodes in parallel according to a rule that one data segment corresponds to one node. Accordingly, the N nodes may receive the N data segments, respectively, and each node is configured to receive one data segment.
For example, referring to FIG. 1E, R1P1, R1P2, and R1P3 may be sent in parallel to node 1, node 2, and node 3.
Step 103: an ith node of the N nodes receives an ith data segment of the N data segments.
For the ith node of the N nodes, the ith node may receive the ith data segment of the N data segments. The i is greater than or equal to 1 and less than or equal to the N.
For example, when N is 3, and the client splits the target request data into 3 data segments, a first node of the 3 nodes may receive the first data segment of the 3 data segments, a second node may receive the second data segment, and a third node may receive the third data segment.
Step 104: and the ith node sends the ith data segment to other nodes in parallel and receives other data segments respectively sent by other nodes.
The other nodes are nodes except the ith node in the N nodes, and the other data segments are data segments except the ith data segment in the N data segments.
The ith node may simultaneously and respectively send the received ith data segment to other nodes, so that other nodes can respectively receive the ith data segment, and accordingly, the i nodes can also receive other data segments sent by other nodes. That is, the N nodes may respectively send the received data segments to other data segments, so as to mutually copy the received data segments, so that each node can receive the N data segments included in the target request data.
For example, referring to fig. 1E, node 1 may send the received R1P1 to nodes 2 and 3 in parallel, node 2 may send the received R1P2 to nodes 1 and 2 in parallel, and node 3 may send the received R1P3 to nodes 1 and 2 in parallel. Accordingly, node 1 may receive R1P2 sent by node 2 and R1P3 sent by node 3, respectively, node 2 may receive R1P1 sent by node 1 and R1P2 sent by node 2, respectively, and node 3 may receive R1P1 sent by node 1 and R1P2 sent by node 2, respectively.
Further, after receiving the data segment sent by the other node from the other node, an acknowledgement response may also be sent to the other node, where the acknowledgement response is used to indicate that the sent data segment has been received, so as to avoid the other node from repeatedly sending the data segment.
In the related art, as the number of nodes, that is, the number of copies increases, although the reliability of the system increases, the number of copies that the Leader node needs to wait for increases, the delay increases, the system performance decreases, and the efficiency of data copying is low. Moreover, since all the requested data needs to be copied through the Leader node, the Leader node becomes a bottleneck of data copying when the data volume exceeds the bearing capacity of the Leader node.
In the embodiment of the application, the target request data is segmented by the client according to the number of the nodes, the segmented data segments are sent to the N nodes in parallel, and the data segments sent by the client are received by the nodes and then are copied in parallel, so that a mechanism for self-adaptively and dynamically segmenting streaming request data and copying copies and activating is realized, the problem of time delay increase caused by copy increase can be effectively solved, and the data copying efficiency is improved. Moreover, each node can receive the data segment and the synchronous data segment at the same time, so that the problem that a single Leader node becomes a performance bottleneck is avoided.
Further, when each of the N data segments carries the identifier of the target request data, after the ith node receives the ith data segment sent by the client in parallel, the ith node may also determine N index flag bits of the target request data according to the identifier of the target request data carried by the ith data segment, and change the ith index flag bit of the N index flag bits from the first flag bit to the second flag bit. The N index flag bits are in one-to-one correspondence with the N data segments, an initial flag bit of the N index flag bits is a first flag bit and is used for indicating that the corresponding data segment is not received, and the second flag bit is used for indicating that the corresponding data segment is received.
Correspondingly, after receiving other data segments respectively sent by the other nodes, the ith node may further change other index flag bits from the first flag bit to the second flag bit, where the other index flag bits are index flag bits other than the ith index flag bit in the N index flag bits.
That is, each time a data segment in the target request data is received, the index flag bit corresponding to the data segment in the N index flag bits may be changed from the original first flag bit to the second flag bit, so as to indicate that the corresponding data segment has been received by the changed index flag bit.
Wherein, the first flag bit and the second flag bit can be preset. For example, the first flag bit may be 0, and the second flag bit may be 1. Assuming that the first flag bit is 0 and the second flag bit is 1, when the ith data segment is received, the ith index flag bit of the N index flag bits may be changed from 0 to 1, and the ith data segment may be correspondingly stored.
The N index flag bits may be obtained by the i-th node according to the N node settings in advance, or may be obtained by the i-th node after receiving the i-th data segment according to the identifier of the target request data carried by the i-th data segment and the N node settings.
In an embodiment, the ith node may set a Hash Map in advance according to the N nodes, where the Hash Map includes N index flag bits corresponding to multiple request data to be received, and after receiving an ith data fragment carrying a target request data identifier, may select N index identifier bits from the N index flag bits as the N index flag bits corresponding to the target request data.
In another embodiment, the ith node may also set N index flag bits for the target request data after receiving the ith data segment carrying the target request data identifier, instead of setting the index flag bit in advance.
For example, referring to fig. 1E, assuming that the client continuously sends 2 target request data, where the 2 target request data are Requst1(R1) and Requst2(R2), respectively, where R1 includes data segments R1P1, R1P2 and R1P3, and R2 includes data segments R2P1, R2P2 and R2P3, referring to fig. 1F, the ith node may include a Hash cache module storing an index flag (IndexFlag) field corresponding to R1 and an IndexFlag field corresponding to R2. Wherein, the IndexFlag field corresponding to R1 includes 3 index flags, and the 3 index flags respectively correspond to data segments P1, P2, and P3 of R1, and are used to indicate whether the corresponding data segments have been received. The IndexFlag field to which R2 corresponds also includes 3 index flags, which correspond to data segments P1, P2, and P3 of R2, respectively, for indicating whether the corresponding data segments have been received.
Step 105: the ith node stores the ith data segment and other data segments to complete the replication of the target request data.
Specifically, the storing of the ith data segment and other data segments by the ith node may include the following two implementation manners:
the first implementation mode comprises the following steps: each data segment in the N data segments carries a segment identifier, and the ith node stores the ith data segment and the other data segments according to the segment identifiers of the ith data segment and the other data segments. The segment identifier is obtained by adding the segment identifier to each data segment after the client splits the target request data, and the segment identifier is used for indicating the position of the corresponding data segment in the target request data.
The ith node may determine and determine an arrangement order of the ith data fragment and the other data fragments according to the fragment identifiers of the ith data fragment and the other data fragments, and then store the ith data fragment and the other data fragments according to the arrangement order, so that the N finally stored data fragments may constitute the target request data according to the storage order.
For example, referring to fig. 1F, for the received data fragments R1P1, R1P2, and R1P3, the 3 data fragments may be determined to be arranged in the order of R1P1, R1P2, and R1P3 according to the fragment identifications P1, P2, and P3, and then the 3 data fragments may be stored in sequence according to the arrangement order. Accordingly, for the received data fragments R2P1, R2P2 and R2P3, the 3 data fragments may also be determined to be arranged in the order of R2P1, R2P2 and R2P3 according to the fragment identifications P1, P2 and P3, and then the 3 data fragments are stored in sequence according to the arrangement order.
The second implementation mode comprises the following steps: each data segment in the N data segments carries a segment identifier, the ith node obtains a request receiving sequence, the request receiving sequence is used for indicating a receiving sequence of all request data received by the data replication system, and the all request data include the target request data; and the ith node stores the ith data segment and the other data segments according to the request receiving sequence and the segment identifications of the ith data segment and the other data segments.
When each data segment of the N data segments further carries the identifier of the target request data, the ith node may obtain a request receiving sequence in the following two ways:
1) when the ith node is a Leader node of the N nodes for the target request data, determining a request receiving sequence according to the identifier of the target request data carried by the ith data fragment and the receiving time of the ith data fragment, and the identifiers of other received request data except the identifier of the target request data and the receiving time of the corresponding data fragment, and sending the request receiving sequence to the other nodes in parallel.
The Leader node and the Follower node in the N nodes may be obtained by presetting according to the copy object. When a Leader node receives a data segment of new request data, the identifier of the request data carried by the data segment is stored, then a request receiving sequence is determined according to the identifier of the request data stored before the data segment of the new request data is received and the identifier of the new request data, and the request receiving sequence is sent to other Follower nodes.
For example, referring to fig. 1E, when the ith node is a Leader node of the N nodes for the target request data, and the ith node receives the data segment R1P1 first and then receives the data segment R2P1, the request receiving sequence may be determined to be R1-R2 according to the identifier R1 and R2 of the request data, and the request receiving sequence R1-R2 is used to indicate that the data replication system receives R1 first and then R2. Moreover, the Leader node may also send the request receipt order R1-R2 to node 2 and node 3 in parallel.
Further, referring to fig. 1F, the ith node may further include an Order cache module, where the Order cache module stores a receiving Order of the requested data of the data replication system, and the receiving Order is R1-R2.
2) When the ith node is a fowler node of the N nodes for the target request data, receiving the request receiving sequence sent by a Leader node of the N nodes, where the request receiving sequence is determined by the Leader node according to the identifier of the target request data and the receiving time of the received data fragment carried by the data fragment received from the client, and the identifiers of other received request data except the identifier of the target request data and the receiving time of the corresponding data fragment.
Specifically, the storing, by the ith node, the ith data segment and the other data segments according to the request receiving order and the segment identifiers of the ith data segment and the other data segments may include: the ith node determines the storage position of the target request data according to the request receiving sequence; the ith node determines the arrangement sequence of the ith data segment and the other data segments according to the segment identifiers of the ith data segment and the other data segments; and the ith node stores the ith data segment and the other data segments in the storage position according to the arrangement sequence of the ith data segment and the other data segments.
Wherein the storage location is used to indicate a storage location of the target request data in all received request data. For example, referring to fig. 1F, assuming that the target request data is R1, it may be determined that the storage location of R1 precedes the storage location of R2 according to the request reception order R1-R2, and accordingly, the respective data fragments R1P1, R1P2, and R1P3 of R1 may be sequentially stored before R2.
Further, the ith data segment and the other data segments may be stored in a designated storage medium of the ith node, and the designated storage medium may be a designated file or other storage medium. Furthermore, the storage of the ith data segment and the other data segments after the designated storage medium of the ith node is not limited to the synchronous wait for the successful disk-drop.
In the related art, in order to reduce the dependency on the Leader node in the data replication process, a query replication algorithm is further provided, in which a write (Writer) process is required to be specially responsible for writing request data to each node, and it is required to wait for each node to write successfully before indicating that the request data is successfully replicated. However, since each node writes data through the same Writer process, the sequence between each node must be kept consistent, that is, the same node can only be written with data by one Writer process at the same time, and the sequence of the request data written by each node can be guaranteed to be consistent.
In the embodiment of the application, the Leader node determines the request receiving sequence, and the Leader node can synchronize the data segments to the fowler node and simultaneously transmit the request receiving sequence to the fowler node, so that each node can store the request data according to the same request receiving sequence, thereby ensuring that the sequence of the request data stored by each node is consistent, supporting a plurality of clients to simultaneously transmit the request data to the N nodes, namely supporting the plurality of clients to simultaneously write the data, and solving the problem of expansibility of write operation.
Further, the ith node may further store the ith data segment and the other data segments when each index flag bit of the N index flag bits is the second flag bit.
For example, referring to fig. 1E and fig. 1F, the ith node may check the IndexFlag fields corresponding to the request data in the Hash cache module according to the request receiving sequence by traversing the request receiving sequence in the Order cache module, determine whether all the index flag bits in the corresponding IndexFlag fields are the second index flag bits, and write the data fragments corresponding to the request data into the Log module when all the index flag bits in the corresponding IndexFlag fields are the second index flag bits.
Further, after storing the ith data segment and the other data segments, the ith node may further send a confirmation message to the client, where the confirmation message is used to indicate that the storage of the target request data by the ith node is completed, and when the confirmation message sent by the N nodes is received by the client, it confirms that the replication of the target request data in the data replication system is completed.
The confirmation message is a response to the client, and the confirmation message may further carry an identifier of the target request data, and is used to indicate that the ith node of the client completes storage of the target request data through the identifier of the target request data.
The client may wait for responses from the N nodes after sending N data segments of the target request data in parallel to the N nodes. When receiving the confirmation messages sent by the N nodes, the data replication system can confirm that the target request data is completely replicated in the data replication system.
Further, after the client sends N data segments of the target request data to the N nodes in parallel, when an acknowledgment message sent by the N nodes is received within a preset time period, the client may continue to send other request data to the N nodes, when an acknowledgment message sent by the N nodes is not received within the preset time period, it indicates that a response is overtime, the N nodes have failed to copy the target request data, and at this time, the client needs to send the N data segments to the N nodes in parallel again.
In order to describe the data replication efficiency of the data replication method provided in the embodiments of the present application, the replication time of the master-slave replication method provided in the related art and the replication time of the data replication method provided in the embodiments of the present application are described in the following.
Fig. 1G is a schematic flowchart of a master-slave replication method provided in the related art, and as shown in fig. 1G, the master-slave replication method includes the following steps:
1. the client sends request data to the Leader node.
2. And the Leader node writes the received request data into the partition file.
3. The Leader node sends the request data to the Follower.
4. The Follower node writes the received request data into a partition file
5. And the Follower node sends a confirmation message to the Leader node, and the confirmation message indicates that the requested data is successfully written.
6. The Leader node sends a confirmation message to the client to indicate that the requested data is copied successfully.
If TSemd represents the transmission delay of the request data, Twrite represents the write delay of the request data, and Tack represents the transmission delay of the acknowledgement message, the time t1 used in step 1 is TSemd, the time t2 used in step 2 is Twrite, the time t3 used in step 3 is TSemd, the time t4 used in step 4 is Twrite, the time t5 used in step 5 is Tack, and the time t6 used in step 6 is Tack. Therefore, the total time t of the master-slave copy method is t1+ t2+ t3+ t4+ t5+ t6 is 2TSemd +2Twrite +2 Tack.
Fig. 1H is a schematic flowchart of a data replication method according to an embodiment of the present application, and as shown in fig. 1H, the data replication method includes the following steps:
1. the client sends N data segments of the request data to N nodes in parallel.
2. And each node sends the received data fragments to other nodes in parallel.
3. And each node stores the received N data fragments.
4. Each node sends an acknowledgement message to the client indicating that the storage of the requested data is complete.
Since step 1 sends 1/N of the requested data, the time T1 used in step 1 is 1/N TSemd, and similarly, T2 is 1/N TSemd. T3 ═ Twrite and T4 ═ Tack. Therefore, the total time T of the data copying method provided in the embodiment of the present application is T1+ T2+ T3+ T4 is 2/N TSemd + Twrite + Tack.
As can be seen from the above comparative analysis, the total time of the master-slave copy method is 2 to 3 times of the total time of the data copy method provided in the embodiment of the present application, so that the data copy method provided in the embodiment of the present application can effectively reduce the data copy time delay, the system performance is significantly improved, and the improvement of the system performance increases with the increase of the number of copies.
In the embodiment of the application, the client can segment target request data to be sent according to node data included in the data replication system, and then send N data segments obtained by segmentation to N nodes included in the data replication system in parallel. For the ith node of the N nodes, the ith node may receive the ith data fragment of the N data fragments, then send the ith data fragment to other nodes in parallel, receive other data fragments respectively sent by other nodes, and then may complete the replication of the target request data by storing the ith data fragment and other data fragments. The data segment is divided and then parallelly sent to each node, each node receives the corresponding data segment and then copies the data segment, so that the data copying efficiency can be improved, the system performance is improved, and moreover, because the copying of all request data does not need to pass through a Leader node first, the problem that the Leader node becomes a performance bottleneck can be avoided.
Fig. 2 is a schematic structural diagram of a data replication apparatus according to an embodiment of the present application, where the apparatus may be an ith node in N nodes included in a data replication system, and as shown in fig. 2, the apparatus includes: a receiving module 201, a sending module 202 and a storing module 203.
A receiving module 201, configured to perform the steps performed in step 103 in the embodiment described in fig. 1D above;
a sending module 202, configured to execute the sending step executed in step 104 in the embodiment described in fig. 1D above;
the receiving module 201 is further configured to execute the receiving step executed in step 104 in the embodiment described in fig. 1D;
a storage module 203, configured to execute the steps executed in step 105 in the embodiment described above in fig. 1D.
Optionally, each data fragment of the N data fragments carries a fragment identifier, where the fragment identifier is obtained by adding the fragment identifier to each data fragment after the client splits the target request data, and the fragment identifier is used to indicate a position of the corresponding data fragment in the target request data;
the storage module 203 is configured to execute the second implementation manner in step 105 described in the embodiment of fig. 1D.
Optionally, the storage module 203 includes an obtaining unit, and the obtaining unit is configured to execute step 1) or 2) in the second implementation manner included in step 105 described in the embodiment of fig. 1D.
Optionally, the storage module 203 comprises a storage unit, and the storage unit is configured to:
determining the storage position of the target request data according to the request receiving sequence;
determining the arrangement sequence of the ith data segment and the other data segments according to the segment identifications of the ith data segment and the other data segments;
and storing the ith data segment and the other data segments in the storage position according to the arrangement sequence of the ith data segment and the other data segments.
Optionally, each data fragment of the N data fragments carries an identifier of the target request data;
the device also includes:
a determining module, configured to determine, according to the identifier of the target request data carried by the ith data segment, N index flag bits of the target request data, where the N index flag bits are in one-to-one correspondence with the N data segments, and an initial flag bit of the N index flag bits is a first flag bit, and is used to indicate that a corresponding data segment is not received;
a first changing module, configured to change an ith index flag bit of the N index flag bits from the first flag bit to a second flag bit, where the second flag bit is used to indicate that a corresponding data segment has been received;
a second changing module, configured to change, after the receiving module receives other data segments respectively sent by the other nodes, other index bits from the first bit to the second bit, where the other index bits are index bits other than the ith index bit in the N index bits;
the storage module 203 is configured to store the ith data segment and the other data segments when each index flag bit of the N index flag bits is the second flag bit.
Optionally, the sending module 202 is further configured to:
after the storage module 203 stores the ith data segment and the other data segments, a confirmation message is sent to the client, where the confirmation message is used to indicate that the storage of the target request data by the ith node is completed, and when the confirmation message sent by the N nodes is received by the client, the completion of the replication of the target request data in the data replication system is confirmed.
In the embodiment of the application, the client can segment target request data to be sent according to node data included in the data replication system, and then send N data segments obtained by segmentation to N nodes included in the data replication system in parallel. For the ith node of the N nodes, the ith node may receive the ith data fragment of the N data fragments, then send the ith data fragment to other nodes in parallel, receive other data fragments respectively sent by other nodes, and then may complete the replication of the target request data by storing the ith data fragment and other data fragments. The data segment is divided and then parallelly sent to each node, each node receives the corresponding data segment and then copies the data segment, so that the data copying efficiency can be improved, the system performance is improved, and moreover, because the copying of all request data does not need to pass through a Leader node first, the problem that the Leader node becomes a performance bottleneck can be avoided.
It should be noted that: in the data copying provided by the above embodiments, only the division of the functional modules is illustrated when data copying is performed, and in practical applications, the function distribution may be completed by different functional modules according to needs, that is, the internal structure of the apparatus is divided into different functional modules to complete all or part of the functions described above. In addition, the data copying apparatus and the data copying method provided by the above embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments and are not described herein again.
In another embodiment, a computer-readable storage medium is provided, which has instructions stored therein, and when the computer-readable storage medium runs on a computer, the computer is caused to execute the data replication method described in the embodiment of fig. 1D.
In another embodiment, a computer program product containing instructions is provided, which when run on a computer, causes the computer to perform the data replication method described above in the embodiment of fig. 1D.
In the above embodiments, the implementation may be wholly or partly realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored on a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website, computer, server, or data center to another website, computer, server, or data center via wire (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., Digital Versatile Disk (DVD)), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The above-mentioned embodiments are provided not to limit the present application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.
Claims (12)
1. A data replication method applied to a data replication system including a client and N nodes, the method comprising:
receiving, by an ith node of the N nodes, an ith data segment of N data segments sent by the client side in parallel, where the N data segments are obtained by the client side by segmenting target request data to be sent according to the N nodes, where i is greater than or equal to 1 and less than or equal to N, and N is a positive integer;
the ith node sends the ith data segment to other nodes in parallel and receives other data segments sent by the other nodes respectively, wherein the other nodes are nodes except the ith node in the N nodes, and the other data segments are data segments except the ith data segment in the N data segments;
the ith node stores the ith data segment and the other data segments to complete the replication of the target request data;
each data fragment in the N data fragments carries an identifier of the target request data; after the ith node in the N nodes receives the ith data segment in the N data segments sent by the client side in parallel, the method further includes:
determining N index zone bits of the target request data according to the identifier of the target request data carried by the ith data segment, wherein the N index zone bits correspond to the N data segments one by one, and the initial zone bit of the N index zone bits is a first zone bit and is used for indicating that the corresponding data segment is not received;
changing an ith index flag bit of the N index flag bits from the first flag bit to a second flag bit, the second flag bit being used to indicate that a corresponding data segment has been received;
after receiving the other data segments respectively sent by the other nodes, the method further includes:
changing other index flag bits from the first flag bit to the second flag bit, the other index flag bits being index flag bits of the N index flag bits except for the ith index flag bit;
the ith node storing the ith data segment and the other data segments, including:
when each index flag bit of the N index flag bits is the second flag bit, the ith node stores the ith data segment and the other data segments.
2. The method of claim 1, wherein each of the N data segments carries a segment identifier, the segment identifier is obtained by adding the segment identifier to each data segment after the client splits the target request data, and the segment identifier is used to indicate a position of a corresponding data segment in the target request data;
the ith node storing the ith data segment and the other data segments, including:
the ith node acquires a request receiving sequence, wherein the request receiving sequence is used for indicating a receiving sequence of all request data received by the data replication system, and the all request data comprise the target request data;
and the ith node stores the ith data segment and the other data segments according to the request receiving sequence and the segment identifications of the ith data segment and the other data segments.
3. The method of claim 2, wherein each of the N data segments further carries an identification of the target request data;
the obtaining, by the ith node, a request receiving order includes:
when the ith node is a master Leader node for the target request data in the N nodes, determining a request receiving sequence according to the identifier of the target request data carried by the ith data segment and the receiving time of the ith data segment, and the identifiers of other received request data except the identifier of the target request data and the receiving time of the corresponding data segment, and sending the request receiving sequence to other nodes in parallel;
when the ith node is a standby-side follow node for the target request data in the N nodes, receiving the request receiving sequence sent by a Leader node in the N nodes, where the request receiving sequence is determined by the Leader node according to the identifier of the target request data carried in the data segment received from the client and the receiving time of the received data segment, and the identifiers of other received request data except the identifier of the target request data and the receiving time of the corresponding data segment.
4. The method of claim 2, wherein the storing of the ith data segment and the other data segments by the ith node based on the request receipt order and segment identifications of the ith data segment and the other data segments comprises:
the ith node determines the storage position of the target request data according to the request receiving sequence;
the ith node determines the arrangement sequence of the ith data segment and the other data segments according to the segment identifiers of the ith data segment and the other data segments;
and the ith node stores the ith data segment and the other data segments in the storage position according to the arrangement sequence of the ith data segment and the other data segments.
5. The method of any of claims 1-4, wherein after the storing of the ith data segment and the other data segments by the ith node, further comprising:
and the ith node sends a confirmation message to the client, wherein the confirmation message is used for indicating that the ith node finishes storing the target request data, and when the client receives the confirmation messages sent by the N nodes, the client confirms that the target request data is completely copied in the data copying system.
6. A data replication apparatus applied to an ith node among N nodes included in a data replication system including a client and the N nodes, the apparatus comprising:
a receiving module, configured to receive an ith data segment of N data segments sent by the client in parallel, where the N data segments are obtained by the client by segmenting target request data to be sent according to the N nodes, i is greater than or equal to 1 and less than or equal to N, and N is a positive integer;
a sending module, configured to send the ith data segment to other nodes in parallel, where the other nodes are nodes other than the ith node in the N nodes;
the receiving module is further configured to receive other data segments respectively sent by the other nodes, where the other data segments are data segments of the N data segments other than the ith data segment;
the storage module is used for storing the ith data segment and the other data segments so as to complete the replication of the target request data;
each data fragment in the N data fragments carries an identifier of the target request data; the device further comprises:
a determining module, configured to determine, according to an identifier of the target request data carried by the ith data segment, N index flag bits of the target request data, where the N index flag bits correspond to the N data segments one to one, and an initial flag bit of the N index flag bits is a first flag bit, and is used to indicate that a corresponding data segment is not received;
a first changing module, configured to change an ith index flag bit of the N index flag bits from the first flag bit to a second flag bit, where the second flag bit is used to indicate that a corresponding data segment has been received;
a second changing module, configured to change, after the receiving module receives other data segments respectively sent by other nodes, other index flag bits from the first flag bit to the second flag bit, where the other index flag bits are index flag bits of the N index flag bits except for the ith index flag bit;
the storage module is configured to store the ith data segment and the other data segments when each index flag bit of the N index flag bits is the second flag bit.
7. The apparatus of claim 6, wherein each of the N data segments carries a segment identifier, the segment identifier is obtained by adding to each data segment after the client splits the target request data, and the segment identifier is used to indicate a position of a corresponding data segment in the target request data;
the memory module includes:
an obtaining unit, configured to obtain a request receiving order, where the request receiving order is used to indicate a receiving order of all request data that have been received by the data replication system, where the all request data include the target request data;
and the storage unit is used for storing the ith data segment and the other data segments according to the request receiving sequence and the segment identifications of the ith data segment and the other data segments.
8. The apparatus of claim 7, wherein each of the N data segments further carries an identification of the target request data;
the obtaining unit is specifically configured to:
when the ith node is a master Leader node for the target request data in the N nodes, determining a request receiving sequence according to the identifier of the target request data carried by the ith data segment and the receiving time of the ith data segment, and the identifiers of other received request data except the identifier of the target request data and the receiving time of the corresponding data segment, and sending the request receiving sequence to other nodes in parallel;
when the ith node is a standby-side follow node for the target request data in the N nodes, receiving the request receiving sequence sent by a Leader node in the N nodes, where the request receiving sequence is determined by the Leader node according to the identifier of the target request data carried in the data segment received from the client and the receiving time of the received data segment, and the identifiers of other received request data except the identifier of the target request data and the receiving time of the corresponding data segment.
9. The apparatus of claim 7, wherein the storage unit is specifically configured to:
determining the storage position of the target request data according to the request receiving sequence;
determining the arrangement sequence of the ith data segment and the other data segments according to the segment identifications of the ith data segment and the other data segments;
and storing the ith data segment and the other data segments in the storage position according to the arrangement sequence of the ith data segment and the other data segments.
10. The apparatus of any of claims 6-9, wherein the sending module is further configured to:
after the storage module stores the ith data segment and the other data segments, sending a confirmation message to the client, wherein the confirmation message is used for indicating that the ith node finishes storing the target request data, and when the client receives the confirmation messages sent by the N nodes, the client confirms that the target request data is completely copied in the data copying system.
11. A node device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor is configured to perform the steps of any of the methods of claims 1-5.
12. A computer-readable storage medium having instructions stored thereon, which when run on a computer, cause the computer to perform the method of any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711482591.0A CN109992447B (en) | 2017-12-29 | 2017-12-29 | Data copying method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711482591.0A CN109992447B (en) | 2017-12-29 | 2017-12-29 | Data copying method, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109992447A CN109992447A (en) | 2019-07-09 |
| CN109992447B true CN109992447B (en) | 2021-06-15 |
Family
ID=67109234
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711482591.0A Active CN109992447B (en) | 2017-12-29 | 2017-12-29 | Data copying method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109992447B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110868247B (en) * | 2019-11-20 | 2022-02-25 | 哈尔滨工业大学 | Event-driven satellite telemetry data processing method and device and computer storage medium |
| CN111182067B (en) * | 2019-12-31 | 2021-08-27 | 上海焜耀网络科技有限公司 | Data writing method and device based on interplanetary file system IPFS |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101046952A (en) * | 2006-03-29 | 2007-10-03 | 雅马哈株式会社 | Audio network system |
| CN101217678A (en) * | 2007-12-28 | 2008-07-09 | 华为技术有限公司 | System upgrade method, system and device |
| CN101419600A (en) * | 2007-10-22 | 2009-04-29 | 深圳市亚贝电气技术有限公司 | Data copy mapping method and device based on object-oriented LANGUAGE |
| CN101534205A (en) * | 2008-03-11 | 2009-09-16 | 中国网通集团宽带业务应用国家工程实验室有限公司 | Application layer multicast service realizing method, terminal and system thereof |
| CN103873173A (en) * | 2014-03-27 | 2014-06-18 | 北京悦迪文化传媒有限公司 | Digital pushing radio (DPR) method based on wireless digital radio technology |
| CN105406999A (en) * | 2015-11-20 | 2016-03-16 | 福建星网锐捷安防科技有限公司 | Multi-data distribution method and distribution equipment |
-
2017
- 2017-12-29 CN CN201711482591.0A patent/CN109992447B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101046952A (en) * | 2006-03-29 | 2007-10-03 | 雅马哈株式会社 | Audio network system |
| CN101419600A (en) * | 2007-10-22 | 2009-04-29 | 深圳市亚贝电气技术有限公司 | Data copy mapping method and device based on object-oriented LANGUAGE |
| CN101217678A (en) * | 2007-12-28 | 2008-07-09 | 华为技术有限公司 | System upgrade method, system and device |
| CN101534205A (en) * | 2008-03-11 | 2009-09-16 | 中国网通集团宽带业务应用国家工程实验室有限公司 | Application layer multicast service realizing method, terminal and system thereof |
| CN103873173A (en) * | 2014-03-27 | 2014-06-18 | 北京悦迪文化传媒有限公司 | Digital pushing radio (DPR) method based on wireless digital radio technology |
| CN105406999A (en) * | 2015-11-20 | 2016-03-16 | 福建星网锐捷安防科技有限公司 | Multi-data distribution method and distribution equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109992447A (en) | 2019-07-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11445019B2 (en) | Methods, systems, and media for providing distributed database access during a network split | |
| CN109683826B (en) | Capacity expansion method and device for distributed storage system | |
| CN112470112B (en) | Distributed copy of a block storage system | |
| US10382380B1 (en) | Workload management service for first-in first-out queues for network-accessible queuing and messaging services | |
| US9170892B2 (en) | Server failure recovery | |
| US11966307B2 (en) | Re-aligning data replication configuration of primary and secondary data serving entities of a cross-site storage solution after a failover event | |
| CN110058959B (en) | Data backup method, apparatus and computer program product | |
| US10831741B2 (en) | Log-shipping data replication with early log record fetching | |
| US9262323B1 (en) | Replication in distributed caching cluster | |
| US9286298B1 (en) | Methods for enhancing management of backup data sets and devices thereof | |
| US9367261B2 (en) | Computer system, data management method and data management program | |
| CN112084258A (en) | Data synchronization method and device | |
| CN113168404B (en) | System and method for replicating data in a distributed database system | |
| US20190140902A1 (en) | Centralized configuration data in a distributed file system | |
| CN105493474B (en) | System and method for supporting partition level logging for synchronizing data in a distributed data grid | |
| CN109582213B (en) | Data reconstruction method and device and data storage system | |
| CN113268472B (en) | Distributed data storage system and method | |
| CN112052230B (en) | Multi-machine room data synchronization method, computing device and storage medium | |
| CN106873902B (en) | File storage system, data scheduling method and data node | |
| CN109714430A (en) | Distributed caching method, device, computer system and storage medium | |
| US10545667B1 (en) | Dynamic data partitioning for stateless request routing | |
| CN109992447B (en) | Data copying method, device and storage medium | |
| CN111225003B (en) | NFS node configuration method and device | |
| CN113259476B (en) | Message pushing method and system | |
| CN106951443B (en) | Method, equipment and system for synchronizing copies based on distributed system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200424 Address after: 518129 Bantian HUAWEI headquarters office building, Longgang District, Guangdong, Shenzhen Applicant after: HUAWEI TECHNOLOGIES Co.,Ltd. Address before: 301, A building, room 3, building 301, foreshore Road, No. 310052, Binjiang District, Zhejiang, Hangzhou Applicant before: Huawei Technologies Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |