CN111625601A - Data writing method and device, computer equipment and storage medium - Google Patents
Data writing method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN111625601A CN111625601A CN202010463909.6A CN202010463909A CN111625601A CN 111625601 A CN111625601 A CN 111625601A CN 202010463909 A CN202010463909 A CN 202010463909A CN 111625601 A CN111625601 A CN 111625601A
- Authority
- CN
- China
- Prior art keywords
- data
- node
- missing
- data node
- request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/273—Asynchronous replication or reconciliation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1464—Management of the backup or restore process for networked environments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- Quality & Reliability (AREA)
- Computer Security & Cryptography (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present disclosure provides a data writing method, an apparatus, a computer device, and a storage medium, wherein the data writing method is applied to a first data node in a database system, and the method includes: when a first write-in request sent by a second data node is received, whether the data sent from the second data node in the stored data of the first data node is missing or not can be determined according to a request number carried in the first write-in request, and in response to the situation that the data is missing, a target data node in which the missing data is stored can be determined based on the stored data information broadcasted by each data node, and the missing data is obtained from the target data node and stored.
Description
Technical Field
The present disclosure relates to the field of data storage technologies, and in particular, to a data writing method and apparatus, a computer device, and a storage medium.
Background
Currently, in a masterless Database (leader Database), any data node can directly respond to a write request from a user. In recent years, masterless databases such as Dynamo, Riak, Cassandra, voldest, scyldb have received popularity from a large number of users due to their high availability.
Generally, most of the masterless databases compromise consistency among three factors of consistency, availability and partition fault tolerance, and the consistency is an important index of the databases, so that how to keep data consistent among data nodes in the masterless databases is a topic worthy of study.
Disclosure of Invention
The embodiment of the disclosure at least provides a data writing method, a data writing device, computer equipment and a storage medium, which can realize the consistency of stored data in each data node in a database system.
In a first aspect, an embodiment of the present disclosure provides a data writing method, which is applied to a first data node in a database system, where the data writing method includes:
receiving a first write request sent by a second data node in the database system; the first write-in request carries data requested to be written in and a request number corresponding to the data;
determining whether there is a lack of data sent from the second data node in the stored data of the first data node based on the request number;
in response to the situation that the data is missing, determining a target data node storing the missing data based on stored data information broadcast by each data node in the database system;
and acquiring the missing data from the target data node and writing the missing data.
In an optional implementation manner, the data writing method further includes:
deleting data except for the most recently written data in the plurality of data according to the time stamps of the plurality of data aiming at the plurality of data with the same key in the stored data; wherein the data is stored in the form of key-value pairs.
In an optional implementation manner, the data writing method further includes:
receiving a second write-in request sent by the user side;
and writing the data corresponding to the second write request, and sending the data corresponding to the second write request to other data nodes in the database system.
In an optional implementation manner, the second write request carries a configuration value; after the sending the data corresponding to the second write request to other data nodes in the database system, the data writing method further includes:
counting the number of data nodes in which the data corresponding to the second write request is written;
if the number is larger than or equal to the configuration value, feeding back success information of writing data corresponding to the second writing request to the user side;
and if the quantity is smaller than the configuration value, feeding back failure information for writing data corresponding to the second writing request to the user side.
In an optional embodiment, the determining, based on the request number, whether there is a missing of data sent from the second data node in the data already stored by the first data node includes:
determining a maximum data number corresponding to the stored data sent by the second data node;
and determining whether the data sent from the second data node in the stored data of the first data node is missing according to the difference value between the request number and the maximum data number.
In an optional implementation manner, the determining, based on the stored data information broadcast by each data node in the database system, a target data node storing the missing data includes:
and if the maximum continuous number corresponding to the data with the node identification of the second data node in the stored data information broadcasted by any data node is greater than the maximum data number, determining that the data node is the target data node with the missing data.
In an optional implementation manner, the stored data information broadcasted by any data node includes:
the maximum serial number corresponding to the data received by the data node from the user side and the maximum serial number corresponding to the data received by the data node from other data nodes.
In an optional implementation manner, a node identifier corresponding to the maximum consecutive number of data received by any data node from a user side is an identifier of the data node; and the node identification corresponding to the maximum continuous number of the data received by any data node from other data nodes is the identification of the other data nodes.
In an optional embodiment, the obtaining and writing the missing data from the target data node includes:
sending a data acquisition request to the target data node; the data acquisition request carries a request number corresponding to the missing data;
and receiving and writing the missing data sent by the target data node.
In a second aspect, an embodiment of the present disclosure further provides a data writing apparatus, which is applied to a first data node in a database system, where the data writing apparatus includes:
the receiving module is used for receiving a first write request sent by a second data node in the database system; the first write-in request carries data requested to be written in and a request number corresponding to the data;
a first determining module, configured to determine whether there is a missing of data sent from the second data node in the stored data of the first data node based on the request number;
the second determining module is used for responding to the situation that the data is missing, and determining a target data node which stores the missing data based on the stored data information broadcast by each data node in the database system;
and the acquisition module is used for acquiring the missing data from the target data node and writing the missing data.
In an optional implementation manner, the data writing apparatus further includes a deleting module:
the deleting module is used for deleting data except the most recently written data in the plurality of data according to the time stamps of the plurality of data aiming at the plurality of data with the same key in the stored data; wherein the data is stored in the form of key-value pairs.
In an optional implementation, the receiving module is further configured to:
receiving a second write-in request sent by the user side;
and writing the data corresponding to the second write request, and sending the data corresponding to the second write request to other data nodes in the database system.
In an optional implementation manner, the second write request carries a configuration value; the data writing device further comprises a statistic module, wherein the statistic module is used for:
counting the number of data nodes in which the data corresponding to the second write request is written;
if the number is larger than or equal to the configuration value, feeding back success information of writing data corresponding to the second writing request to the user side;
and if the quantity is smaller than the configuration value, feeding back failure information for writing data corresponding to the second writing request to the user side.
In an alternative embodiment, the first determining module includes:
a first determining unit, configured to determine a maximum data number corresponding to the stored data sent by the second data node;
and the second determining unit is used for determining whether the data sent from the second data node in the stored data of the first data node is missing or not according to the difference value between the request number and the maximum data number.
In an optional implementation manner, the second determining module is configured to determine the target data node according to the following steps:
and if the maximum continuous number corresponding to the data with the node identification of the second data node in the stored data information broadcasted by any data node is greater than the maximum data number, determining that the data node is the target data node with the missing data.
In an optional implementation manner, the stored data information broadcasted by any data node includes:
the maximum serial number corresponding to the data received by the data node from the user side and the maximum serial number corresponding to the data received by the data node from other data nodes.
In an optional implementation manner, a node identifier corresponding to the maximum consecutive number of data received by any data node from a user side is an identifier of the data node; and the node identification corresponding to the maximum continuous number of the data received by any data node from other data nodes is the identification of the other data nodes.
In an optional implementation, the obtaining module includes:
a sending unit, configured to send a data acquisition request to the target data node; the data acquisition request carries a request number corresponding to the missing data;
and the receiving unit is used for receiving and writing the missing data sent by the target data node.
In a third aspect, an embodiment of the present disclosure further provides a computer device, including: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating via the bus when the computer device is running, the machine-readable instructions when executed by the processor performing the steps of the first aspect described above, or any possible implementation of the first aspect.
In a fourth aspect, this disclosed embodiment further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and the computer program, when executed by a processor, performs the steps in the first aspect described above or any possible implementation manner of the first aspect.
According to the data writing method, the data writing device, the computer equipment and the storage medium, whether missing data occurs or not can be found in time through the request numbers, the data nodes storing the missing data can be determined through the broadcasted stored data information, and then the missing data is supplemented.
In order to make the aforementioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly described below, and the drawings herein incorporated in and forming a part of the specification illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the technical solutions of the present disclosure. It is appreciated that the following drawings depict only certain embodiments of the disclosure and are therefore not to be considered limiting of its scope, for those skilled in the art will be able to derive additional related drawings therefrom without the benefit of the inventive faculty.
Fig. 1 shows a flowchart of a data writing method provided by an embodiment of the present disclosure;
FIGS. 2a and 2b are schematic diagrams illustrating data completion in an embodiment of the disclosure;
FIG. 3 is a flow chart illustrating another data writing method provided by the embodiments of the present disclosure;
FIG. 4 is a schematic diagram of a data writing apparatus provided by an embodiment of the present disclosure;
FIG. 5 is a schematic diagram of another data writing apparatus provided by an embodiment of the present disclosure;
fig. 6 shows a specific schematic diagram of a first determining module in the data writing device provided in the embodiment of the present disclosure;
fig. 7 shows a specific schematic diagram of an obtaining module in the data writing device according to the embodiment of the disclosure;
fig. 8 shows a schematic diagram of a computer device provided by an embodiment of the present disclosure.
Illustration of the drawings: 400-data writing means; 410-a receiving module; 420-a first determination module; 421-a first determination unit; 422-a second determination unit; 430-a second determination module; 440-an acquisition module; 441-a sending unit; 442-a receiving unit; 450-delete module; 460-a statistics module; 800-a computer device; 801-a processor; 802-a memory; 8021-memory; 8022 external memory, 803-bus.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure. It is to be understood that the described embodiments are merely illustrative of some, and not restrictive, of the embodiments of the disclosure. The components of the embodiments of the present disclosure, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure, presented in the figures, is not intended to limit the scope of the claimed disclosure, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the disclosure without making creative efforts, shall fall within the protection scope of the disclosure.
According to research, most of the unowned databases compromise consistency among three factors of consistency, availability and partition fault tolerance, and degrade the unowned databases into a final consistency system, but the systems still face the following problems:
data loss problem, generally, data forwarded by one data node can normally reach other data nodes in the unowned database, but in the weak network environment, there are still situations that data cannot reach other data nodes, data loss is inevitable, and how is the data stored between the data nodes the same and complete during data loss?
Here, the industry aims at the following scheme adopted by no master database to solve the problem that the data loss causes data inconsistency stored by each data node:
and (3) a data reading restoration scheme, specifically, by using the principle that W + R > N, namely the sum of the number W of successful writing and the number R of successful reading is greater than the number N of data nodes, a user reads data from a plurality of data nodes, and restores the data after comparing differences. However, in some scenarios where extremely high performance and extremely high availability are sought, users tend to abandon the configuration of W + R > N, and instead become successful in writing data to a data node, i.e., writing-reading-scenario, and successful in reading data from a data node, in which scenario the data read from a single data node cannot be judged to be up-to-date and valid, so the repair scheme when reading data fails in this scenario.
Based on the above research, the present disclosure provides a data writing method, an apparatus, a computer device, and a storage medium, wherein when a first write request sent by a second data node is received, according to a request number carried in the first write request, whether a missing data occurs in data sent from the second data node in data stored in the first data node can be determined, and in response to the missing data, a target data node storing the missing data can be determined based on stored data information broadcast by each data node, and the missing data is obtained from the target data node and stored, so that whether the missing data occurs can be timely found through the request number, and the data node storing the missing data can be determined through the broadcasted stored data information, and then the missing data is filled up, and consistency of the stored data in each data node in a database system can be achieved, moreover, the method does not need to occupy more software and hardware resources, and can better solve the problem of inconsistent data in a write-read scene.
The solutions set forth in the present disclosure are all results of inventors after practical and careful study, and therefore, the discovery process of the above problems and the solutions proposed by the present disclosure hereinafter for the above problems should be contributions to the present disclosure.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
To facilitate understanding of the present embodiment, first, a data writing method disclosed in the embodiments of the present disclosure is described in detail, where an execution subject of the data writing method provided in the embodiments of the present disclosure is generally a computer device with certain computing capability, and the computer device includes, for example: a terminal device, which may be a User Equipment (UE), a mobile device, a User terminal, a cellular phone, a wireless phone, a Personal Digital Assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, or a server or other processing device. In some possible implementations, the data writing method may be implemented by a processor calling computer readable instructions stored in a memory.
The following describes a data writing method provided by the embodiment of the present disclosure, taking an execution subject as a terminal device as an example.
Example one
Referring to fig. 1, a flowchart of a data writing method provided by an embodiment of the present disclosure is shown, where the data writing method is applied to a first data node, and the method includes steps S101 to S104, where:
s101: receiving a first write request sent by a second data node in the database system; the first write request carries data requested to be written and a request number corresponding to the data.
In this step, the first data node receives a first write request sent by a second data node in the database system, where the first write request is a request from a user side forwarded by the second data node, and the first write request carries data requested to be written and a request number corresponding to the data.
It should be noted that, a database system includes a plurality of data nodes, each data node backs up with each other, different data nodes should store the same data theoretically, each data node in the database system has the same function, and each data node is not different in nature, here, each data node can receive a write request sent by other data nodes in the database system, and the write request is a request from a user end forwarded by the other data nodes, that is, each data node can indirectly receive the write request sent by the user end; each data node can also receive a write request or a read request directly sent by a user side, wherein the database system is a master-less database.
Here, when a data node receives a write request sent by a user side, data requested to be written is numbered, and generally, the number corresponding to each data is determined according to the number of requests of the write request sent by the user side, which is received by the data node. The data nodes are allowed to number data carried in the received write-in requests sent from the user side only, and the data nodes are not allowed to number data carried in the received write-in requests sent from other data nodes, so that the same data are prevented from being numbered twice.
Here, the numbering rule is shown in the following formula:
where x is the number of requests, f (x) is the number corresponding to the number of requests x, a is the numbering step between two numbers, a is a positive integer, and for convenience of numbering, preferably, a is 1.
It should be further noted that each data node includes a coordination layer, a replication layer, and a storage layer, wherein all read requests and write requests sent by a user side directly reach the coordination layer in the data node, and then are forwarded to the replication layer in the data node and replication layers of other data nodes by the coordination layer; the replication layer is used for receiving data sent by the coordination layer belonging to the same data node and data sent by the coordination layer in other data nodes; the storage layer is used for storing data and reading data from the storage layer by a user end, and comprises a journal sub-module (WAL) and a storage engine sub-module.
The first data node may also directly receive a write request and a read request sent by a user side, and specifically, after receiving a second write request sent by the user side, the first data node numbers and writes data corresponding to the second write request, and sends the data corresponding to the second write request to other data nodes in the database system, where the second write request sent by the first data node to other data nodes carries numbers corresponding to data requested to be written.
Generally, for a write request, it can be set that success information can be fed back to a user as long as one data node is successfully written. Specifically, the first data node encodes the data in the second write request and then sends the encoded data to the replication layer and other data nodes in the data node, and after any one node returns success, the data can be fed back to the user.
Further, for the write request, the second write request of the user may carry a configuration value, so that the first data node needs to wait until the accumulated number of the successes returned by other nodes or the node reaches the configuration value and then feeds back the successes to the user, otherwise, the first data node fails to feed back the successes to the user. Namely, after the first data node sends the data corresponding to the second write request to other data nodes in the database system, the method further comprises the following steps:
counting the number of data nodes in which the data corresponding to the second write request is written; if the number is larger than or equal to the configuration value, feeding back success information of writing data corresponding to the second writing request to the user side; and if the quantity is smaller than the configuration value, feeding back failure information for writing data corresponding to the second writing request to the user side.
In the step, the first data node counts the number of data nodes, including the first data node, which successfully write in the data carried by the second data request, and when the number is judged to be greater than or equal to the configuration value, the first data node feeds back success information of writing in the data corresponding to the second write request to the user side; and feeding back a failure message for writing data corresponding to the second writing request to the user side when the quantity is judged to be smaller than the configuration value, wherein the configuration value is the quantity of successful writing. Specifically, the coordination layer in the first data node may collect, according to the configuration value given by the user, the response number returned by each data node in the database system for the second write request, and if the response number reaches the configuration value, it indicates that the number of data nodes that have been successfully written reaches the configuration value.
S102: and determining whether the data sent from the second data node in the stored data of the first data node is missing or not based on the request number.
In this step, the first data node determines whether there is data from the second data node in its stored data according to a request number corresponding to data carried in a first write request sent by the second data node, where the data sent by the second data node and received by the first data node all have the request number and have a node identifier of the second data node.
It should be noted that, in general, data forwarded by one data node may normally reach other data nodes in the database system, but under the conditions of a weak network environment, a network failure, and the like, there may be a case that the data cannot reach other data nodes, that is, there is a data loss problem, and therefore, data loss is unavoidable.
Here, if the number of data nodes in the database system is N, the complexity of the numbering rule is o (a), and the spatial complexity is o (N), since the number of data nodes in the database system is usually very small (within 16, generally 3), the space-time complexity for determining whether a missing data scheme occurs tends to a constant level through numbering, and the space-time complexity is low, so that less software and hardware resources are required.
Further, in step S102, determining whether there is a missing of the data sent from the second data node in the stored data of the first data node based on the request number includes the following steps:
determining a maximum data number corresponding to the stored data sent by the second data node; and determining whether the data sent from the second data node in the stored data of the first data node is missing according to the difference value between the request number and the maximum data number.
In this step, the first data node determines, according to the node identifier of the second data node, data corresponding to the node identifier with the second data node from the stored data, and further determines, from numbers corresponding to the data with the node identifier of the second data node, a number with a maximum number value, that is, a maximum data number, and further determines, according to a difference between a request number corresponding to data carried in the first write request and the maximum data number, whether data sent by the second data node is absent from the stored data, specifically, if the difference is greater than a number step, it is determined that the data sent by the second data node is absent; and if the difference is less than or equal to the numbering step, determining that the data sent by the second data node is not lacked. According to the method and the device, each data sent by the user side is numbered, and whether missing data occurs is determined by using the numbers, so that whether the missing data occurs can be determined simply and efficiently.
S103: and in response to the situation that the data is missing, determining a target data node storing the missing data based on the stored data information broadcasted by each data node in the database system.
In this step, the first data node determines that there is lack of data sent by the second data node in its own stored data, that is, it determines that there is a case where there is no data sent by the second data node, it needs to complete the missing data, and here, each data node in the database system broadcasts its own stored data information in real time or at preset time intervals, so the first data node determines a target data node storing the missing data according to the received stored data information broadcast by other data nodes, where the target data node and the second data node are the same data node, or other data nodes except the first data node and the second data node, and there may be a plurality of target data nodes having the missing data.
It should be noted that the replication layer in each data node is used for data synchronization in each data node in the database system, for example, when a fault such as a weak network or a network partition occurs, a write request sent by the coordination layer of another data node may be lost midway, which further causes a situation of data loss, and when the replication layer detects data loss, the replication layer may compare data in real time and complete the missing data.
Further, the following explains a process of determining a target data node having missing data, that is, determining the target data node having the missing data stored therein based on the stored data information broadcast by each data node in the database system in step S103, and includes the following steps:
and if the maximum continuous number corresponding to the data with the node identification of the second data node in the stored data information broadcasted by any data node is greater than the maximum data number, determining that the data node is the target data node with the missing data.
In this step, if the maximum consecutive number corresponding to the data with the node identifier of the second data node in the stored data information broadcast by any data node is greater than the maximum data number, it is determined that the data node is the target data node with missing data, otherwise, it is determined that the data node is not the target data node with missing data. The maximum consecutive number is a number having the largest number value among a plurality of numbers having consecutive numbers.
Here, the stored data information broadcast by any data node includes: the maximum serial number corresponding to the data received by the data node from the user side and the maximum serial number corresponding to the data received by the data node from other data nodes.
It should be noted that, in order to improve the broadcasting efficiency, only the number of a part of data and the node identifier corresponding to the data may be broadcast, but it is required to ensure that when other data nodes acquire the stored data information, it can be determined whether missing data is stored in the data nodes broadcasting the stored data information, specifically, the stored data information broadcast by any data node includes the maximum serial number corresponding to the data received by the data node from the user side and the maximum serial number corresponding to the data received by the data node from other data nodes.
It should be noted that the purpose of broadcasting the stored data information by each data node is to enable other data nodes to determine whether there is missing data in the broadcast stored information, and if so, to conveniently complete the missing data, so as to ensure that there is no missing data in the storage layer in the data node.
The node identification corresponding to the maximum continuous number of the data received by any data node from the user side is the identification of the data node; and the node identification corresponding to the maximum continuous number of the data received by any data node from other data nodes is the identification of the other data nodes.
S104: and acquiring the missing data from the target data node and writing the missing data.
In the step, when the first data node determines that the target data node has the own missing data, the first data node acquires the missing data from the target data node and stores the missing data, so that consistency of data stored in each data node in the database system can be realized by finding the missing data in time and supplementing the missing data.
Here, considering recovery of abnormal states such as downtime and hard disk failure, data that is not sent to other data nodes in time may be lost, for example, in the process of distributing requests by a coordination layer in a data node, because the speed of sending to a replication layer of the coordination layer is very fast (memory is taken) and the speed of sending to replication layers of other data nodes is slow (network is taken), there may be a case that a request sent to the coordination layer is successfully responded and returned to a user terminal, and a request sent to other data nodes is not sent from the data node in time, and a downtime failure occurs at this time, and then the data is lost.
It should be noted that, for the coordination layer and the replication layer of the same data node, the data sent by the coordination layer to the replication layer does not have the situation that the replication layer cannot receive the data, that is, the data transmission in the same process is not lost, and the data sent by the coordination layer to the replication layer in other data nodes has the situation that the data transmission fails and the data loss occurs due to a network failure and poor network signal. Therefore, by adopting the method for detecting data missing in real time and supplementing the missing data in the disclosure, the data nodes storing the missing data are determined by continuously broadcasting the stored data information along with the data nodes, and the missing data is acquired from the data nodes to achieve the purpose of supplementing the data, so that the data sets stored by all the data nodes are finally the same.
Further, the step S104 of obtaining the missing data from the target data node and writing the missing data includes the following steps:
sending a data acquisition request to the target data node; the data acquisition request carries a request number corresponding to the missing data; and receiving and writing the missing data sent by the target data node.
In this step, the first data node sends a data acquisition request to the target data node, where the data acquisition request carries a request number corresponding to the missing data, and further, according to the request number of the missing data and the node identifier of the second data node, the target data node can know the missing data to be acquired by the first data node, and then the target data node sends the missing data to the first data node. To this end, the first data node completes the process from the detection of the missing data to the acquisition of the missing data.
As shown in fig. 2a and fig. 2B, schematic diagrams of data completion in the embodiment of the present disclosure are shown, and here, an execution flow of data completion in the present scheme is described in a specific embodiment, where a database system includes a data node a, a data node B, and a data node C, a numbering step is 1, and the execution flow includes the following steps:
step (1): when the data node a receives a write request sent by the data node B, the data B3 carried by the request in the write request and the request number 3 corresponding to the data are acquired.
Step (2): after the data node a receives the data node B3 sent by the data node B, as shown in fig. 2a, it may determine that the maximum data number corresponding to the data node B stored by the data node a is 3, the difference between 3 and 1 is 2, and 2 is greater than the number step 1, and then it is determined that the data node a lacks the data that the data node B should write into the data node a, i.e. lacks the data node B2 sent by the data node B and having the number 2.
And (3): as shown in fig. 2a, both data node a and data node B may receive the stored data information broadcast by data node C, where the maximum consecutive number of the data with node id B is 3, and 3 is greater than 2, which indicates that there is missing data B2 of data node a in data node C.
And (4): as shown in FIG. 2a, data node A sends a get request for data B2 to data node C and gets the missing data B2 from data node C.
Here, fig. 2b illustrates an implementation flow of data padding, including the following steps:
step (1), the data node C broadcasts self storage information to the data node A and the data node B;
step (2), after receiving the storage information broadcast by the data node C, the data node A and the data node B judge whether the data node C has the data missing by themselves; here, the data node a determines that the data node C has data missing by itself, and the data node B determines that the data node C does not have data missing by itself;
step (3), the data node A sends a request for acquiring self missing data to the data node C;
and (4) after receiving the request of the data node A, the data node C sends the missing data to the data node A.
In the embodiment of the disclosure, when a first write request sent by a second data node is received, according to a request number carried in the first write request, whether a missing occurs in data sent from the second data node in data stored in the first data node can be determined, in response to the existence of the missing, based on stored data information broadcast by each data node, a target data node storing the missing data can be determined, and the missing data is obtained from the target data node and stored, so that whether the missing data occurs can be timely found through the request number, and through broadcasted stored data information, the data node storing the missing data can be determined, and then the missing data is supplemented, and consistency of data stored in each data node in a database system can be realized, where consistency refers to making data sets stored in all data nodes in the database system identical, which may be understood as the integrity of the data.
Example two
Referring to fig. 3, a flowchart of another data writing method provided in the embodiment of the present disclosure is shown, where the method includes steps S301 to S305, where:
s301: receiving a first write request sent by a second data node in the database system; the first write request carries data requested to be written and a request number corresponding to the data.
S302: and determining whether the data sent from the second data node in the stored data of the first data node is missing or not based on the request number.
S303: and in response to the situation that the data is missing, determining a target data node storing the missing data based on the stored data information broadcasted by each data node in the database system.
S304: and acquiring the missing data from the target data node and writing the missing data.
S305: deleting data except for the most recently written data in the plurality of data according to the time stamps of the plurality of data aiming at the plurality of data with the same key in the stored data; wherein the data is stored in the form of key-value pairs.
In this step, in a scenario of concurrent writing, a problem of data collision may occur, that is, one data node may store multiple data of the same key at the same time, for this reason, when receiving data, a timestamp is assigned to each data, and the timestamp corresponding to each data can represent the time of writing the data, and further, according to the timestamps of multiple data with the same key, data other than the most recently written data in the multiple data is deleted, that is, only the most recently written data is retained.
Here, the present disclosure is directed to a method for handling a data collision problem, which is simple and efficient, by deleting data other than the most recently written data among a plurality of stored data, based on time stamps of the plurality of data, so that the problem of data collision can be solved by using the time stamps, and finally, data in each data node can be made uniform.
As will be explained in detail below, usually, a time sequence relationship exists between data sent by the same data node, but in the entire database system, data sent by the coordination layers of all the data nodes may fall into the storage layers of each data node in a staggered manner, and a global time sequence relationship does not exist between data in the storage layer of each data node, so that conflicting data cannot be directly merged. In order to determine the global timing relationship, the timestamp is added to the data at the coordination layer of the data node or the user side, the timestamp is encoded based on the real time, and the timestamp includes the real time and may also include the identifier of the data node.
Here, the time stamp may be encoded according to table 1, and the encoding specification of the time stamp is shown in table 1:
TABLE 1 encoded details of timestamps
| Name (R) | Length (bit) | Description of the invention |
| Real time | 56 | High position |
| Reserved field | 2 | Median, reserved field |
| Data node identification | 6 | Low order, data node identification |
The reserved field is filled with 0 under the default condition, because the node identifiers of all the data nodes are different, the timestamps formed by the real time, the reserved field and the node identifiers of the data nodes are globally unique and have a partial order relationship, the data with the timestamps are stored, and the subsequent conflict combination can be conveniently carried out.
In an example, in a scenario where Key X exists in a certain data node, at a certain moment or within a certain period of time with a sufficiently small interval, both data node a and data node B receive a modification request for Key X, which is labeled as ma (X) and mb (X), considering factors such as network delay in a data synchronization process, data modification sequences acquired by data node a and data node B, that is, the sequence of ma (X) and mb (X), are likely to be opposite, that is, different, and in such a scenario, data collision occurs between data node a and data node B. The background periodic repair scheme is a bottom-entering scheme, and is generally difficult to accept by users, because the background process periodically compares the data of all data nodes with the data of all increments, detects missing, fills or merges conflicts, and finally makes the data among the data nodes consistent.
For the present disclosure, since the replication layers in each data node perform real-time data completion, the data sets that finally fall into the storage layers in each data node are the same, but there may be data with the same key written at different times in the data sets, that is, there is a data conflict.
According to the data consistency method, data differences are compared among data nodes in a database system, missing data are filled up, written data sets of all the data nodes are finally the same, furthermore, each data comprises a globally unique timestamp with a partial sequence relation, when all the data nodes stop writing and a period of time passes, each data node respectively processes conflict data stored by the data node, and consistency of the data in all the data nodes in the database system can be achieved.
According to the data processing method and device, each data node can automatically find data missing and complete in real time, data conflict is processed by taking the timestamp as a basis, and the data processing method and device have the advantages of being high in performance, low in delay, small in data inconsistency window and the like in a write-read-one use mode. Moreover, the database system disclosed by the invention has the characteristic of high robustness, the change operations such as switching master and the like are not required when the network partition fails, the reading and writing of each partition can still be smoothly carried out, and the missing data can be automatically supplemented after the network partition failure is recovered.
The descriptions of steps S301 to S304 can refer to the descriptions of steps S101 to S104, and the same technical effects can be achieved, and are not further explained here.
In the embodiment of the disclosure, when a first write request sent by a second data node is received, according to a request number carried in the first write request, whether a data sent from the second data node is missing in a stored data of the first data node can be determined, and in response to the situation that the data is missing, based on stored data information broadcast by each data node, a target data node in which the missing data is stored can be determined, and the missing data is obtained from the target data node and stored.
It will be understood by those skilled in the art that in the method of the present invention, the order of writing the steps does not imply a strict order of execution and any limitations on the implementation, and the specific order of execution of the steps should be determined by their function and possible inherent logic.
Based on the same inventive concept, a data writing device corresponding to the data writing method is also provided in the embodiments of the present disclosure, and because the principle of solving the problem of the device in the embodiments of the present disclosure is similar to the data writing method in the embodiments of the present disclosure, the implementation of the device may refer to the implementation of the method, and repeated details are not repeated.
EXAMPLE III
Referring to fig. 4, which is a schematic diagram of a data writing apparatus 400 according to an embodiment of the present disclosure, the data writing apparatus 400 includes: a receiving module 410, a first determining module 420, a second determining module 430, and an obtaining module 440; wherein the content of the first and second substances,
a receiving module 410, configured to receive a first write request sent by a second data node in the database system; the first write-in request carries data requested to be written in and a request number corresponding to the data;
a first determining module 420, configured to determine whether there is a missing of data sent from the second data node in the stored data of the first data node based on the request number;
a second determining module 430, configured to determine, in response to a situation that there is a missing data, a target data node in which the missing data is stored based on stored data information broadcast by each data node in the database system;
an obtaining module 440, configured to obtain the missing data from the target data node and write the missing data.
In an embodiment of the present disclosure, fig. 5 illustrates a schematic diagram of another data writing apparatus 400 provided by an embodiment of the present disclosure; the data writing device 400 further includes a deleting module 450:
the deleting module 450 is configured to delete, according to timestamps of a plurality of data with the same key in stored data, data other than the most recently written data in the plurality of data; wherein the data is stored in the form of key-value pairs.
In an embodiment of the present disclosure, as shown in fig. 4 and 5, the receiving module 410 is further configured to:
receiving a second write-in request sent by the user side;
and writing the data corresponding to the second write request, and sending the data corresponding to the second write request to other data nodes in the database system.
In the embodiment of the present disclosure, as shown in fig. 5, the second write request carries a configuration value; the data writing apparatus 400 further includes a statistics module 460, the statistics module 460 is configured to:
counting the number of data nodes in which the data corresponding to the second write request is written;
if the number is larger than or equal to the configuration value, feeding back success information of writing data corresponding to the second writing request to the user side;
and if the quantity is smaller than the configuration value, feeding back failure information for writing data corresponding to the second writing request to the user side.
In an embodiment of the present disclosure, fig. 6 illustrates a specific schematic diagram of a first determining module 420 in a data writing apparatus 400 provided in an embodiment of the present disclosure; the first determination module 420 includes:
a first determining unit 421, configured to determine a maximum data number corresponding to the stored data sent by the second data node;
a second determining unit 422, configured to determine whether there is a missing of the data sent from the second data node in the stored data of the first data node according to the difference between the request number and the maximum data number.
In an embodiment of the present disclosure, as shown in fig. 4 and fig. 5, the second determining module 430 is configured to determine the target data node according to the following steps:
and if the maximum continuous number corresponding to the data with the node identification of the second data node in the stored data information broadcasted by any data node is greater than the maximum data number, determining that the data node is the target data node with the missing data.
In an embodiment of the present disclosure, the stored data information broadcasted by any data node includes:
the maximum serial number corresponding to the data received by the data node from the user side and the maximum serial number corresponding to the data received by the data node from other data nodes.
In an embodiment of the present disclosure, a node identifier corresponding to a maximum consecutive number of data received by any data node from a user side is an identifier of the data node; and the node identification corresponding to the maximum continuous number of the data received by any data node from other data nodes is the identification of the other data nodes.
In an embodiment of the present disclosure, fig. 7 illustrates a specific schematic diagram of an obtaining module 440 in a data writing apparatus 400 provided in an embodiment of the present disclosure;
the obtaining module 440 includes:
a sending unit 441, configured to send a data obtaining request to the target data node; the data acquisition request carries a request number corresponding to the missing data;
a receiving unit 442, configured to receive and write the missing data sent by the target data node.
In the embodiment of the disclosure, when a first write request sent by a second data node is received, according to a request number carried in the first write request, whether a data sent from the second data node is missing in a stored data of the first data node can be determined, and in response to the situation that the data is missing, based on stored data information broadcast by each data node, a target data node in which the missing data is stored can be determined, and the missing data is obtained from the target data node and stored.
The description of the processing flow of each module in the device and the interaction flow between the modules may refer to the related description in the above method embodiments, and will not be described in detail here.
Example four
Based on the same technical concept, the embodiment of the disclosure also provides computer equipment. Referring to fig. 8, a schematic structural diagram of a computer device 800 provided in the embodiment of the present disclosure includes a processor 801, a memory 802, and a bus 803. The memory 802 is used for storing execution instructions and includes a memory 8021 and an external memory 8022; the memory 8021 is also referred to as an internal memory, and is used for temporarily storing operation data in the processor 801 and data exchanged with an external storage 8022 such as a hard disk, the processor 801 exchanges data with the external storage 8022 through the memory 8021, and when the computer apparatus 800 operates, the processor 801 communicates with the storage 802 through the bus 803, so that the processor 801 executes the following instructions:
in one possible implementation, the instructions executed by the processor 801,
receiving a first write request sent by a second data node in the database system; the first write-in request carries data requested to be written in and a request number corresponding to the data;
determining whether there is a lack of data sent from the second data node in the stored data of the first data node based on the request number;
in response to the situation that the data is missing, determining a target data node storing the missing data based on stored data information broadcast by each data node in the database system;
and the target data node acquires the missing data from the target data node and writes the missing data.
EXAMPLE five
The embodiments of the present disclosure also provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program performs the steps of the data writing method in the above method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.
The computer program product of the data writing method provided in the embodiments of the present disclosure includes a computer-readable storage medium storing a program code, where instructions included in the program code may be used to execute the steps of the data writing method in the above method embodiments, which may be referred to specifically for the above method embodiments, and are not described herein again.
The embodiments of the present disclosure also provide a computer program, which when executed by a processor implements any one of the methods of the foregoing embodiments. The computer program product may be embodied in hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK), or the like.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present disclosure may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present disclosure. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
Finally, it should be noted that: the above-mentioned embodiments are merely specific embodiments of the present disclosure, which are used for illustrating the technical solutions of the present disclosure and not for limiting the same, and the scope of the present disclosure is not limited thereto, and although the present disclosure is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive of the technical solutions described in the foregoing embodiments or equivalent technical features thereof within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present disclosure, and should be construed as being included therein. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.
Claims (12)
1. A data writing method applied to a first data node in a database system, the data writing method comprising:
receiving a first write request sent by a second data node in the database system; the first write-in request carries data requested to be written in and a request number corresponding to the data;
determining whether there is a lack of data sent from the second data node in the stored data of the first data node based on the request number;
in response to the situation that the data is missing, determining a target data node storing the missing data based on stored data information broadcast by each data node in the database system;
and acquiring the missing data from the target data node and writing the missing data.
2. The data writing method according to claim 1, further comprising:
deleting data except for the most recently written data in the plurality of data according to the time stamps of the plurality of data aiming at the plurality of data with the same key in the stored data; wherein the data is stored in the form of key-value pairs.
3. The data writing method according to claim 1, further comprising:
receiving a second write-in request sent by the user side;
and writing the data corresponding to the second write request, and sending the data corresponding to the second write request to other data nodes in the database system.
4. The data writing method according to claim 3, wherein the second write request carries a configuration value; after the sending the data corresponding to the second write request to other data nodes in the database system, the data writing method further includes:
counting the number of data nodes in which the data corresponding to the second write request is written;
if the number is larger than or equal to the configuration value, feeding back success information of writing data corresponding to the second writing request to the user side;
and if the quantity is smaller than the configuration value, feeding back failure information for writing data corresponding to the second writing request to the user side.
5. The data writing method according to claim 1, wherein the determining whether there is a missing of data sent from the second data node in the data stored in the first data node based on the request number comprises:
determining a maximum data number corresponding to the stored data sent by the second data node;
and determining whether the data sent from the second data node in the stored data of the first data node is missing according to the difference value between the request number and the maximum data number.
6. The data writing method according to claim 5, wherein the determining a target data node storing the missing data based on the stored data information broadcast by each data node in the database system comprises:
and if the maximum continuous number corresponding to the data with the node identification of the second data node in the stored data information broadcasted by any data node is greater than the maximum data number, determining that the data node is the target data node with the missing data.
7. The data writing method according to claim 6, wherein the stored data information broadcasted by any data node includes:
the maximum serial number corresponding to the data received by the data node from the user side and the maximum serial number corresponding to the data received by the data node from other data nodes.
8. The data writing method according to claim 7, wherein the node identifier corresponding to the maximum consecutive number of the data received by any data node from the user side is the identifier of the data node; and the node identification corresponding to the maximum continuous number of the data received by any data node from other data nodes is the identification of the other data nodes.
9. The data writing method according to claim 1, wherein the obtaining and writing the missing data from the target data node comprises:
sending a data acquisition request to the target data node; the data acquisition request carries a request number corresponding to the missing data;
and receiving and writing the missing data sent by the target data node.
10. A data writing apparatus, applied to a first data node in a database system, the data writing apparatus comprising:
the receiving module is used for receiving a first write request sent by a second data node in the database system; the first write-in request carries data requested to be written in and a request number corresponding to the data;
a first determining module, configured to determine whether there is a missing of data sent from the second data node in the stored data of the first data node based on the request number;
the second determining module is used for responding to the situation that the data is missing, and determining a target data node which stores the missing data based on the stored data information broadcast by each data node in the database system;
and the acquisition module is used for acquiring the missing data from the target data node and writing the missing data.
11. A computer device, comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating via the bus when a computer device is running, the machine-readable instructions when executed by the processor performing the steps of the data writing method of any one of claims 1 to 9.
12. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps of the data writing method according to any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010463909.6A CN111625601B (en) | 2020-05-27 | 2020-05-27 | Data writing method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010463909.6A CN111625601B (en) | 2020-05-27 | 2020-05-27 | Data writing method and device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111625601A true CN111625601A (en) | 2020-09-04 |
| CN111625601B CN111625601B (en) | 2023-03-14 |
Family
ID=72271584
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010463909.6A Active CN111625601B (en) | 2020-05-27 | 2020-05-27 | Data writing method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111625601B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070180307A1 (en) * | 2003-07-15 | 2007-08-02 | Xiv Limited | Method & system for resynchronizing data between a primary and mirror data storage system |
| US20110208996A1 (en) * | 2010-02-22 | 2011-08-25 | International Business Machines Corporation | Read-other protocol for maintaining parity coherency in a write-back distributed redundancy data storage system |
| CN102402600A (en) * | 2011-11-18 | 2012-04-04 | 国电南瑞科技股份有限公司 | Asynchronous collecting method for maintaining consistency of database replicas |
| CN102970104A (en) * | 2012-10-24 | 2013-03-13 | 北京经纬恒润科技有限公司 | Method for acquiring data and server |
| CN106534261A (en) * | 2016-10-09 | 2017-03-22 | 乐视控股（北京）有限公司 | Data writing method, device and system |
| US20180101329A1 (en) * | 2014-10-29 | 2018-04-12 | International Business Machines Corporation | Strong consistency write threshold |
| CN111163120A (en) * | 2018-11-08 | 2020-05-15 | 阿里巴巴集团控股有限公司 | Data storage and transmission method and device of distributed database and storage medium |
| CN111198662A (en) * | 2020-01-03 | 2020-05-26 | 腾讯科技（深圳）有限公司 | Data storage method and device and computer readable storage medium |
-
2020
- 2020-05-27 CN CN202010463909.6A patent/CN111625601B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070180307A1 (en) * | 2003-07-15 | 2007-08-02 | Xiv Limited | Method & system for resynchronizing data between a primary and mirror data storage system |
| US20110208996A1 (en) * | 2010-02-22 | 2011-08-25 | International Business Machines Corporation | Read-other protocol for maintaining parity coherency in a write-back distributed redundancy data storage system |
| CN102402600A (en) * | 2011-11-18 | 2012-04-04 | 国电南瑞科技股份有限公司 | Asynchronous collecting method for maintaining consistency of database replicas |
| CN102970104A (en) * | 2012-10-24 | 2013-03-13 | 北京经纬恒润科技有限公司 | Method for acquiring data and server |
| US20180101329A1 (en) * | 2014-10-29 | 2018-04-12 | International Business Machines Corporation | Strong consistency write threshold |
| CN106534261A (en) * | 2016-10-09 | 2017-03-22 | 乐视控股（北京）有限公司 | Data writing method, device and system |
| CN111163120A (en) * | 2018-11-08 | 2020-05-15 | 阿里巴巴集团控股有限公司 | Data storage and transmission method and device of distributed database and storage medium |
| CN111198662A (en) * | 2020-01-03 | 2020-05-26 | 腾讯科技（深圳）有限公司 | Data storage method and device and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111625601B (en) | 2023-03-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105138571B (en) | Distributed file system and method for storing massive small files | |
| US20170032007A1 (en) | System and method for augmenting consensus election in a distributed database | |
| US7685185B2 (en) | Move-in/move-out notification for partial replica synchronization | |
| CN102609479B (en) | A kind of memory database node clone method | |
| CN112905390A (en) | Log data backup method, device, equipment and storage medium | |
| US10664349B2 (en) | Method and device for file storage | |
| CN101344882B (en) | Data query method, insertion method and deletion method | |
| CN113297309A (en) | Stream data writing method, device, equipment and storage medium | |
| CN113064764A (en) | Method and apparatus for performing blocks in a blockchain system | |
| CN115146002A (en) | Cross-data-center data synchronization method and device | |
| CN107908713B (en) | Distributed dynamic rhododendron filtering system based on Redis cluster and filtering method thereof | |
| CN107203437B (en) | Method, device and system for preventing memory data from being lost | |
| CN111625601B (en) | Data writing method and device, computer equipment and storage medium | |
| CN107465706B (en) | Distributed data object storage device based on wireless communication network | |
| CN111404737B (en) | Disaster recovery processing method and related device | |
| CN113704359A (en) | Synchronization method, system and server for multiple data copies of time sequence database | |
| CN111625617A (en) | Data indexing method and device and computer readable storage medium | |
| CN116303789A (en) | Parallel synchronization method and device for multi-fragment multi-copy database and readable medium | |
| CN116339626A (en) | Data processing method, device, computer equipment and storage medium | |
| Li et al. | A novel min-process checkpointing scheme for mobile computing systems | |
| CN115017126A (en) | Multi-application log aggregation query system and method | |
| CN114328749A (en) | Business data processing method and device and computer readable storage medium | |
| CN116737764A (en) | Method and device for data synchronization, electronic equipment and storage medium | |
| CN114020836A (en) | Distributed industrial SCADA system measurement data processing method based on time sequence library | |
| CN115442272B (en) | Method, device, equipment and storage medium for detecting lost data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220630 Address after: 100190 1309, 13th floor, building 4, Zijin Digital Park, Haidian District, Beijing Applicant after: Beijing volcano Engine Technology Co.,Ltd. Address before: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Applicant before: BEIJING BYTEDANCE NETWORK TECHNOLOGY Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |