CN116521085A - Log entry submitting method and device, electronic equipment and storage medium - Google Patents

Log entry submitting method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN116521085A
CN116521085A CN202310511899.2A CN202310511899A CN116521085A CN 116521085 A CN116521085 A CN 116521085A CN 202310511899 A CN202310511899 A CN 202310511899A CN 116521085 A CN116521085 A CN 116521085A
Authority
CN
China
Prior art keywords
sequence number
number information
log
copy
data center
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310511899.2A
Other languages
Chinese (zh)
Inventor
刘新宇
王蒙蒙
韩朱忠
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Dameng Database Co Ltd
Original Assignee
Shanghai Dameng Database Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Dameng Database Co Ltd filed Critical Shanghai Dameng Database Co Ltd
Priority to CN202310511899.2A priority Critical patent/CN116521085A/en
Publication of CN116521085A publication Critical patent/CN116521085A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0614—Improving the reliability of storage systems
    • G06F3/0619—Improving the reliability of storage systems in relation to data integrity, e.g. data losses, bit errors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/11—File system administration, e.g. details of archiving or snapshots
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/17—Details of further file system functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
    • G06F16/1815—Journaling file systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/0643—Management of files
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The invention discloses a log entry submitting method, a log entry submitting device, electronic equipment and a storage medium. The method comprises the following steps: determining a copy mapping relationship when submitting the journal entries to be submitted in each round, wherein the copy mapping relationship indicates a logic data center to which each copy node in the multi-copy cluster belongs; determining log sequence number information corresponding to each copy node, wherein the log sequence number information indicates the maximum sequence number of a log entry stored in the corresponding copy node; determining target log sequence number information according to the copy mapping relation and the plurality of log sequence number information, so that at least one copy node of each logic data center stores log entries to be submitted; and submitting the journal entries to be submitted according to the target journal serial number information. The method comprises the steps of submitting the journal entries to be submitted when at least one copy node of each logic data center is guaranteed to store the journal entries to be submitted, and guaranteeing zero loss of data in an application scene of a multi-copy cluster.

Description

Log entry submitting method and device, electronic equipment and storage medium
Technical Field
The embodiment of the invention relates to the technical field of computers, in particular to a method and a device for submitting log entries, electronic equipment and a storage medium.
Background
The multi-copy cluster is a distributed cluster based on RAFT (Replicated And Fault Tolerant) protocol, RAFT is a distributed algorithm, and mainly solves the problem of consistency in distribution, and one data copy can be stored on a plurality of different nodes respectively in a plurality of identical copies, and the plurality of different nodes form the multi-copy cluster. Compared with the traditional data daemon cluster, the multi-copy cluster is independent of daemons and monitors, has the characteristics of automatic copy master selection, automatic fault processing and automatic fault recovery, and the normal operation of the whole system is not affected by few copies which have faults or network delays.
In the RAFT protocol, there are three election roles: LEADER (LEADER), FOLLOWER (FOLLOWER), and CANDIDATE (CANDIDATE). For a log entry, if the log entry is dropped on multiple copies in the multi-copy cluster, the log entry may be committed on the LEADER. Where a log entry is dropped onto a copy is understood to mean that the log entry is kept in the disk of the node that stored the copy.
In practical application, multiple copies can be deployed in machine rooms in different areas, the machine room in each area can be regarded as a physical data center, and if a log entry is only dropped on multiple copies included in one physical data center and meets the condition that the log entry can be submitted on a LEADER, the LEADER can submit the log entry. However, after the LEADER submits the log entry, if all the multiple copies included in the physical data center fail to recover, the log entry submitted in the physical data center is lost, so that zero loss of the data cannot be guaranteed.
Disclosure of Invention
The invention provides a log entry submitting method, a log entry submitting device, electronic equipment and a storage medium, which can ensure zero loss of data in an application scene of a multi-copy cluster.
In a first aspect, an embodiment of the present invention provides a log entry commit method, the method performed by a leader node in a multi-copy cluster, the method comprising:
determining a copy mapping relation when submitting the journal entries to be submitted in each round, wherein the copy mapping relation indicates a logic data center to which each copy node in the multi-copy cluster belongs;
Determining log sequence number information corresponding to each copy node, wherein the log sequence number information indicates the maximum sequence number of a log entry stored in the corresponding copy node;
determining target log sequence number information according to the copy mapping relation and the plurality of log sequence number information, so that at least one copy node of each logical data center stores the log entries to be submitted;
submitting the journal entries to be submitted according to the target journal serial number information;
the logical data centers are data centers to which logically set duplicate nodes belong, the number of the logical data centers is multiple, and each logical data center comprises one or more duplicate nodes.
In a second aspect, an embodiment of the present invention provides a log entry submitting apparatus, including:
the first determining module is used for determining a copy mapping relation when the journal entries to be submitted are submitted in each round, wherein the copy mapping relation indicates a logic data center to which each copy node in the multi-copy cluster belongs;
the second determining module is used for determining the log sequence number information corresponding to each copy node, wherein the log sequence number information indicates the maximum sequence number of the log entries stored in the corresponding copy node;
The third determining module is used for determining target log sequence number information according to the copy mapping relation and the log sequence number information so that at least one copy node of each logic data center stores the log entries to be submitted;
the submitting module is used for submitting the journal entries to be submitted according to the target journal serial number information;
the logical data centers are data centers to which logically set duplicate nodes belong, the number of the logical data centers is multiple, and each logical data center comprises one or more duplicate nodes.
In a third aspect, an embodiment of the present invention provides an electronic device, including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the method according to the first aspect.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, implements a method as described in the first aspect.
According to the technical scheme, when the journal entries to be submitted are submitted in each round, a copy mapping relation is determined, and the copy mapping relation indicates a logical data center to which each copy node in the multi-copy cluster belongs; determining log sequence number information corresponding to each copy node, wherein the log sequence number information indicates the maximum sequence number of a log entry stored in the corresponding copy node; determining target log sequence number information according to the copy mapping relation and the plurality of log sequence number information, so that at least one copy node of each logic data center stores log entries to be submitted; and submitting the journal entries to be submitted according to the target journal serial number information. According to the technical scheme, when at least one copy node of each logic data center is guaranteed to store the journal entries to be submitted, the journal entries to be submitted are submitted according to the target journal serial number information, and zero loss of data can be guaranteed in an application scene of the multi-copy cluster.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a log entry commit method provided in accordance with a first embodiment of the present invention;
FIG. 2 is a flow chart of a log entry commit method provided in accordance with a second embodiment of the present invention;
FIG. 3 is a flowchart of a method for implementing data integrity of a multi-data center under a multi-copy cluster according to a third embodiment of the present invention;
FIG. 4 is a schematic diagram of a log entry submitting apparatus according to a fourth embodiment of the present invention;
fig. 5 shows a schematic diagram of the structure of an electronic device that may be used to implement an embodiment of the invention.
Detailed Description
In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
It should be noted that the terms "first," "second," and the like herein are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It will be appreciated that before using the technical solutions disclosed in the embodiments of the present invention, the user should be informed and authorized of the type, usage range, usage scenario, etc. of the personal information related to the present disclosure in an appropriate manner according to relevant legal regulations.
Example 1
Fig. 1 is a flowchart of a log entry submission method according to an embodiment of the present invention, where the present embodiment is applicable to a case of performing log entry submission in an application scenario of a multi-copy cluster, and the method may be performed by a log entry submission device, which may be implemented in software and/or hardware and integrated in an electronic device. Further, the electronic device includes, but is not limited to: computers, notebook computers, smart phones, servers, etc. As shown in fig. 1, the method is performed by a leader node in a multi-copy cluster, the method comprising:
S110, determining a copy mapping relation when submitting the journal entries to be submitted in each round, wherein the copy mapping relation indicates a logic data center to which each copy node in the multi-copy cluster belongs.
The to-commit log entry may refer to a log entry that needs to be committed by a leader node in the multi-copy cluster for each round. In the actual use process of the database, the data in the database can be recorded through the log entries, the corresponding data can be recovered or reconstructed through the log entries, and the performance of the database can be improved through the log, so that the log landing can be regarded as a data landing in a multi-copy cluster, wherein the log landing can be understood as that the log entries are landed on a copy, namely, the log entries are stored in a disk of a node storing the copy.
The logical data centers are data centers to which logically set replica nodes belong, the number of the logical data centers is multiple, each logical data center comprises one or more replica nodes, and the replica nodes are nodes for storing replicas. A replica node may be affiliated with only one logical data center or may not belong to any one logical data center, and a logical data center may contain one or more replica nodes.
In practical application, multiple copies can be deployed in machine rooms in different regions (i.e. multiple copies are stored in copy nodes included in the machine rooms in different regions), and the machine room in each region can be regarded as a physical data center, i.e. one physical data center can include one or more copy nodes.
The logic data center can be completely consistent with the configuration of the actual physical data center, or can be different, the logic data center and the physical data center are not associated, and the logic data center can be divided according to the actual physical deployment condition so as to ensure that the condition of data loss cannot exist. For example, for a multi-copy cluster containing 5 copies, the physical data center of a site contains (copy 1, copy 2, and copy 3) 3 copies, and the physical data center of B site contains (copy 4, and copy 5) 2 copies, where the configuration of the logical data center may be identical to that of the physical data center, i.e., copy 1, copy 2, and copy 3 belong to logical data center 1, and copy 4 and copy 5 belong to logical data center 2; the configuration of the logical data center may also be inconsistent with the physical data center, for example, copy 1 belongs to logical data center 1, copy 2 belongs to logical data center 2, copy 3 belongs to logical data center 3, and copies 4 and 5 belong to logical data center 4.
When the journal entry to be submitted is submitted in each round, the mode of determining the copy mapping relationship is not limited, for example, a logical data center to which the copy node depends may be configured for each copy node according to actual application needs, and the corresponding relationship between each copy node and the logical data center to which the copy node depends is determined as the copy mapping relationship, that is, it may be determined by the copy mapping relationship which logical data center each copy node should belong to, or it may also be determined one or more copy nodes included in each logical data center. The copy mapping relationship may be kept unchanged, or the copy mapping relationship may be updated before the journal entry to be submitted is submitted in each round, which is not limited in the present invention.
S120, determining log sequence number information corresponding to each copy node, wherein the log sequence number information indicates the maximum sequence number of the log entries stored in the corresponding copy node.
The log sequence number information indicates the maximum sequence number of the log entry that has been deposited into the corresponding replica node. In practical applications, when multiple copies need to be saved, the leader node transmits corresponding log entries to the multiple copy nodes. The log entries generated by the leader node have globally unique serial numbers, and when the leader node generates the log entries, the serial numbers of the generated log entries can be increased according to the sequence of generating the log entries, for example, the serial numbers of the log entries generated by the leader node can be sequentially 1, 2 and 3. Due to different networks, software, hardware and the like, the timing of receiving log entries transmitted by the leader node by each duplicate node is different, and the writing timing is also different. Some copies write fast and some copies write slow. It may occur that the serial numbers of the largest log entries already stored in each duplicate node are different.
The manner of determining the log sequence number information corresponding to each copy node is not limited, for example, each copy node determines the maximum sequence number of the locally stored log entry as the log sequence number information corresponding to the copy node, and after each copy node determines the corresponding log sequence number information, the leader node can request each copy node to transmit the corresponding log sequence number information; for example, each copy node actively transmits the corresponding log sequence number information to the leader node, so that the leader node can determine the log sequence number information corresponding to each copy node.
In one embodiment, the method further comprises:
and for the log sequence number information corresponding to each copy node, updating the log sequence number information when the corresponding copy node receives the log entry each time.
The leader node uniformly generates the sequence numbers of the log entries, and the duplicate node can passively receive the sequence numbers of the log entries uniformly generated by the leader node, but cannot change the sequence numbers of the received log entries. When each copy node receives the log entry transmitted by the leader node, the log sequence number information stored in each copy node can be updated to the sequence number corresponding to the currently received log entry, namely, the log sequence number information is updated when each copy node receives the log entry.
The leader node may monitor each replica node to determine that each replica node received a log entry. After each copy node updates the log sequence number information, the updated log sequence number information can be transmitted to the leader node, so that the leader node can determine the updated log sequence number information corresponding to the copy node.
In one embodiment, the serial numbers corresponding to the log entries received by the replica node a may be 1, 2, and 3, and after normal writing, the maximum serial number of the log entries stored in the replica node a indicated by the log serial number information corresponding to the replica node a is 3. The serial numbers corresponding to the log entries received by the duplicate node B may be 6, 7, and 8, and after normal writing, the maximum serial number of the log entries stored in the duplicate node B indicated by the log serial number information corresponding to the duplicate node B is 8.
S130, determining target log sequence number information according to the copy mapping relation and the plurality of log sequence number information, so that at least one copy node of each logic data center stores log entries to be submitted.
The target log sequence number information may indicate a maximum sequence number of a log sequence number corresponding to the log entry to be submitted when the log entry to be submitted is submitted.
The method for determining the target log sequence number information is not limited according to the copy mapping relationship and the plurality of log sequence number information, for example, one or more copy nodes contained in each logical data center can be determined according to the copy mapping relationship, the log sequence number information corresponding to each copy node contained in the logical data center is determined in the plurality of log sequence number information, and the maximum sequence number is determined as the central log sequence number information from the sequence numbers indicated by the log sequence number information corresponding to each copy node contained in the logical data center; determining the minimum sequence number in the sequence numbers indicated by the log sequence number information of each center as logic log sequence number information; determining the median of the sequence numbers indicated by the plurality of log sequence number information as physical log sequence number information; and determining the minimum sequence number of the sequence numbers indicated by the logic log sequence number information and the sequence numbers indicated by the physical log sequence number information as target log sequence number information, wherein the target log sequence number information can enable at least one copy node of each logic data center to store log entries to be submitted.
In one embodiment, the replica mapping relationship indicates that the logical data center 1 includes a replica node 1, a replica node 2, and a replica node 3, and the logical data center 2 includes a replica node 4 and a replica node 5; the number indicated by the log sequence number information corresponding to the copy node 1 is 1400, the number indicated by the log sequence number information corresponding to the copy node 2 is 1500, the number indicated by the log sequence number information corresponding to the copy node 3 is 1600, the number indicated by the log sequence number information corresponding to the copy node 4 is 1200, and the number indicated by the log sequence number information corresponding to the copy node 5 is 1300; the serial numbers indicated by the central log serial number information corresponding to the logic data center 1 are the maximum serial numbers 1600 in 1400, 1500 and 1600, and the serial numbers indicated by the central log serial number information corresponding to the logic data center 2 are the maximum serial numbers 1300 in 1200 and 1300; determining the minimum sequence number 1300 in the sequence numbers 1600 and 1300 indicated by the log sequence number information of each center as the sequence number indicated by the logical log sequence number information; determining the median 1400 of the sequence numbers 1400, 1500, 1600, 1200 and 1300 indicated by the plurality of log sequence number information as the sequence number indicated by the physical log sequence number information; the smallest sequence number among the sequence number 1300 indicated by the logical log sequence number information and the sequence number 1400 indicated by the physical log sequence number information is determined as the sequence number indicated by the target log sequence number information.
S140, submitting the journal entries to be submitted according to the target journal serial number information.
The target log sequence number information may indicate a maximum sequence number of a log sequence number corresponding to the log entry to be submitted when the log entry to be submitted is submitted, and the submitting of the log entry to be submitted may be performed when the log sequence number corresponding to the log entry to be submitted is smaller than or equal to the sequence number indicated by the target log sequence number information.
According to the technical scheme, when the journal entries to be submitted are submitted in each round, a copy mapping relation is determined, and the copy mapping relation indicates a logical data center to which each copy node in the multi-copy cluster belongs; determining log sequence number information corresponding to each copy node, wherein the log sequence number information indicates the maximum sequence number of a log entry stored in the corresponding copy node; determining target log sequence number information according to the copy mapping relation and the plurality of log sequence number information, so that at least one copy node of each logic data center stores log entries to be submitted; and submitting the journal entries to be submitted according to the target journal serial number information. According to the technical scheme, when at least one copy node of each logic data center is guaranteed to store the journal entries to be submitted, the journal entries to be submitted are submitted according to the target journal serial number information, and zero loss of data can be guaranteed in an application scene of the multi-copy cluster.
Example two
Fig. 2 is a flowchart of a log entry submitting method according to a second embodiment of the present invention, where, based on the first embodiment, the target log sequence number information is determined to be further refined according to the copy mapping relationship and the plurality of log sequence number information, and the log entry submitted according to the target log sequence number information is submitted to be further refined, as shown in fig. 2, the method includes:
s110, determining a copy mapping relation when submitting the journal entries to be submitted in each round, wherein the copy mapping relation indicates a logic data center to which each copy node in the multi-copy cluster belongs.
S120, determining log sequence number information corresponding to each copy node, wherein the log sequence number information indicates the maximum sequence number of the log entries stored in the corresponding copy node.
S131, according to the copy mapping relation and the plurality of log sequence number information, the central log sequence number information corresponding to each logic data center is determined.
The method for determining the central log sequence number information corresponding to each logical data center is not limited according to the copy mapping relationship and the plurality of log sequence number information, for example, the maximum sequence number in the sequence numbers indicated by the log sequence number information corresponding to the copy nodes contained in each logical data center is determined as the central log sequence number information according to the copy mapping relationship and the plurality of log sequence number information.
Further, determining central log sequence number information corresponding to each logical data center according to the copy mapping relationship and the plurality of log sequence number information, including:
for each logic data center, determining each copy node included in the logic data center according to the copy mapping relation;
determining log sequence number information corresponding to each copy node included in the logic data center in the plurality of log sequence number information;
and according to the determined log sequence number information, determining central log sequence number information, wherein the central log sequence number information indicates the maximum sequence number in the sequence numbers indicated by the log sequence number information.
The copy mapping relationship can determine which logical data center each copy node should belong to, and also can determine one or more copy nodes contained in each logical data center, so that the one or more copy nodes contained in each logical data center can be determined through the copy mapping relationship.
For each logical data center, after determining one or more copy nodes included in the logical data center, the log sequence number information corresponding to each copy node in the one or more copy nodes included in the logical data center is determined in the plurality of log sequence number information, and the maximum sequence number is determined as the central log sequence number information corresponding to the logical data center from the sequence numbers indicated by the log sequence number information corresponding to each copy node included in the logical data center.
S132, determining logic log sequence number information according to the central log sequence number information corresponding to each logic data center.
The mode of determining the logical log sequence number information is not limited according to the central log sequence number information corresponding to each logical data center, and if the minimum sequence number in the sequence numbers indicated by the central log sequence number information is determined as the logical log sequence number information.
Further, determining the logical log sequence number information according to the central log sequence number information corresponding to each logical data center includes:
and determining logic log sequence number information according to the central log sequence number information corresponding to each logic data center, wherein the logic log sequence number information indicates the minimum sequence number in the sequence numbers indicated by the central log sequence number information.
And determining the minimum sequence number in the sequence numbers indicated by the log sequence number information of each center as logic log sequence number information, wherein the sequence number indicated by the logic log sequence number information is the maximum sequence number capable of enabling at least one copy node of each logic data center to store the log entry to be submitted.
S133, determining the median of the serial numbers of the log entries indicated by the plurality of log serial number information as physical log serial number information.
The physical log serial number information indicates serial numbers meeting log entry submitting conditions corresponding to physical data centers, the physical data centers are data centers to which the replica nodes belong in set practical application, the number of the physical data centers is one or more, and each physical data center comprises one or more replica nodes; the journal entry commit condition is that more than half of all the replica nodes included in all the physical data centers can store journal entries to be committed.
And determining the median of the sequence numbers of the log entries indicated by the plurality of log sequence number information as physical log sequence number information, namely, enabling the sequence numbers indicated by the physical log sequence number information to be the maximum sequence numbers of the log entries to be submitted in more than half of all the copy nodes included in all the physical data centers.
S134, determining target log sequence number information according to the logic log sequence number information and the physical log sequence number information.
The method for determining the target log sequence number information according to the logical log sequence number information and the physical log sequence number information is not limited, and if the minimum sequence number of the sequence numbers indicated by the logical log sequence number information and the physical log sequence number information is determined to be the target log sequence number information.
Further, determining target log sequence number information according to the logical log sequence number information and the physical log sequence number information includes:
and determining target log sequence number information according to the logic log sequence number information and the physical log sequence number information, wherein the target log sequence number information indicates the minimum sequence number of the sequence numbers indicated by the logic log sequence number information and the sequence numbers indicated by the physical log sequence number information.
The minimum sequence number of the sequence numbers indicated by the logic log sequence number information and the sequence numbers indicated by the physical log sequence number information is determined to be the target log sequence number information, so that the sequence number indicated by the target log sequence number information can be the maximum sequence number which can enable at least one copy node of each logic data center to store log entries to be submitted, and can enable more than half of all copy nodes included in all physical data centers to store the log entries to be submitted.
S141, submitting the journal entry to be submitted by the target submitting sequence number, wherein the target submitting sequence number does not exceed the sequence number indicated by the target journal sequence number information.
The target log sequence number information may indicate a maximum sequence number of a log sequence number corresponding to the log entry to be submitted when the log entry to be submitted is submitted, and the log entry to be submitted may be submitted with a target submitted sequence number not exceeding the sequence number indicated by the target log sequence number information.
According to the technical scheme, when the journal entries to be submitted are submitted in each round, central journal serial number information corresponding to each logic data center is determined according to the copy mapping relation and the plurality of journal serial number information; determining logic log sequence number information according to the central log sequence number information corresponding to each logic data center; the method comprises the steps of determining the median of serial numbers of log entries indicated by a plurality of pieces of log serial number information as physical log serial number information; according to the logic log sequence number information and the physical log sequence number information, determining target log sequence number information, enabling at least one copy node of each logic data center to store log entries to be submitted according to the sequence number indicated by the target log sequence number information, enabling more than half of all copy nodes included in all the physical data centers to store the maximum sequence number of the log entries to be submitted, and finally submitting the log entries to be submitted according to the target log sequence number information, so that zero loss of data can be guaranteed under the application scene of the multi-copy cluster.
Example III
The embodiments of the present invention are exemplary illustrations of the above embodiments.
In the actual use process of the multi-copy cluster, copies are often deployed in machine rooms in different areas, and the machine room in each area can be regarded as a physical data center (hereinafter, simply referred to as a physical data center). For a multi-copy cluster scenario including 5 copies, the a ground physical data center is configured with three copies, the B ground physical data center is configured with two copies, and under most commit rules of the RAFT algorithm (i.e. log entry commit conditions corresponding to the physical data center), log commit and transaction commit can be performed on LEADER nodes after the log falls over more than half of the copies.
The following scenario may occur at this time: a log package (i.e., log entry) has been dropped on three copies of the physical data center of a but not on two copies of the physical data center of B, when a commit condition is met, a LEADER may commit the log package. If the LEADER submits the log packet, the physical data center of the land a fails, and all the three copies of the land a fail to recover, the log packet submitted in the land a is lost, so that the recovery point target (Recovery point objective, RPO) cannot be guaranteed to be 0, that is, the loss of the data 0 cannot be guaranteed, wherein RPO can represent the amount of data loss which can be tolerated by the service system when a disaster or an emergency occurs. In practical applications of multi-copy clusters, this problem is not acceptable to users with strict data loss requirements.
The embodiment of the invention provides a method for realizing the data integrity of a multi-data center under a multi-copy cluster, and aims to provide a method for solving the problem of data loss in the scene, wherein the concept of a logic data center is introduced to improve the majority of submission rules of a RAFT protocol so as to ensure that the requirement of RPO=0 can be met under the scene that a physical data center fails and all copies in the physical data center cannot be recovered under the scene of multi-place physical data center configuration.
On the basis of introducing a logic data center, most of submission rules of the RAFT protocol are improved, and the submission rules of the logic data center are increased: for a log package, it is not only necessary that the log package be dropped onto more than half of the replicas, but it is also necessary that the log package have at least one replica dropped in each logical data center (even if at least one replica node of each logical data center has a log entry to be committed), and the log package can be committed.
Under the above-described commit rules, it may be ensured that the committed journal package has at least one copy at each logical data center. In this way, for the scenario that the multi-copy cluster is deployed in multiple places, the configuration of the logical data centers and the physical data centers can be consistent, if the data is not lost, it is only required to ensure that at least one active copy exists in each logical center and can participate in log submission, that is, at least one copy falls off from each logical data center when each log packet is submitted, the submitted log packet can be ensured to exist on at least one copy in each logical data center, and meanwhile, the submitted log packet is ensured to exist on at least one copy in the corresponding physical data center. Even if all copies of a certain physical data center fail, the submitted log packet can be found on other physical data centers, and the loss condition can not occur.
The following are illustrated:
for a multi-replica cluster containing 5 replicas, replica 1, replica 2, replica 3 belongs to physical data center 1, replica 4, and replica 5 belongs to physical data center 2. The logical data center and the physical data center are planned to be completely consistent, namely, a copy 1, a copy 2, a copy 3 belongs to the logical data center 1, a copy 4 and a copy 5 belongs to the logical data center 2.
For a certain log packet, if the log packet is to be submitted, at least 3 copies need to be dropped, and at least one copy of each logical data center needs to be dropped, that is, on the premise that most copies are dropped, at least one of the log packet in the copy 1, the copy 2 and the copy 3 needs to be dropped, and at least one of the copy 4 and the copy 5 needs to be dropped. When a certain log packet is submitted, even if the physical data center 1 crashes, all three copies of the physical data center 1 fail, and the log packet can be found on at least one copy (namely, the copy 4 or the copy 5) in the physical data center 2, so that the problem of data loss caused by all the failures of all the copies in a certain physical data center of a multi-copy cluster is solved.
Fig. 3 is a flowchart of a method for implementing data integrity of a multi-data center under a multi-copy cluster according to a third embodiment of the present invention, where, as shown in fig. 3, specific implementation steps are as follows:
(1) The copy is started, and the mapping relation (namely, the copy mapping relation) is initialized. The method comprises the steps of configuring a logic data center sequence number for each copy, and initializing a logic data center sequence number and a copy mapping relation in a memory after the copy is started.
(2) Caching the mapping relation when the central configuration is unchanged, preparing log submission, and obtaining the mapping relation; collecting copy disc brushing information, namely acquiring all copy disc landing information (namely determining log sequence number information corresponding to each copy node), and preparing log submission.
(3) The submittable locations are calculated according to data center rules (i.e., logical data center submission rules). Firstly collecting the largest log entries of all copies in a tray (namely determining log sequence number information corresponding to each copy node) at regular intervals by a LEADER, obtaining the largest log entries of the tray in each logic data center according to the mapping relation between the copies and the logic data center, and taking the entries as the log entries of the logic data center (namely determining the central log sequence number information corresponding to each logic data center according to the copy mapping relation and a plurality of log sequence number information), wherein the aim of the step is to determine the largest log entries which can be submitted by each logic data center; the LEADER counts the log entries of all the logic data centers, then selects the log entry with the smallest sequence number of all the logic data centers, and the corresponding log entry is the largest submittable log entry under the rule submitted by the logic data center (namely, the logic log sequence number information is determined according to the center log sequence number information corresponding to each logic data center).
(4) The submittable locations are calculated according to majority submittal rules. I.e., the master (i.e., the leader node) counts the median of the largest log entry that all copies have fallen (i.e., the median of the sequence numbers of the log entries indicated by the plurality of log sequence number information, determined as physical log sequence number information).
(5) Determining the maximum submittable position, namely determining the maximum submittable position according to the submittable position of the data center rule and the submittable position of the majority of the submittable rules, comparing the submittable position of the majority of the submittable rules with the maximum submittable journal entries under the logic data center submittable rule, taking a small value between the two as the position of the master library which can be submitted at the time (namely determining target journal sequence number information according to logic journal sequence number information and physical journal sequence number information, and submitting journal entries to be submitted according to the target journal sequence number information).
(6) And starting the next round of log submission, and determining the maximum submittable position again according to the latest landing information for submission.
The following are illustrated:
for a multi-copy cluster containing 5 copies, the data center is configured as follows:
physical data center 1 (replica 1, replica 2, replica 3);
physical data center 2 (replica 4, replica 5).
The logical data center and the physical data center are configured consistently, and the configuration of the serial number information (the numbers in brackets represent serial number values) of the dropped log is as follows:
logical data center 1: copy 1, log dropped sequence number SEQ1[1400]; copy 2, log dropped sequence number SEQ2[1500]; copy 3, log dropped sequence number SEQ3[1600];
logical data center 2: copy 4, log dropped sequence number SEQ4[1200]; copy 5, log has fallen to disk number SEQ5[1300].
(1) In a first step, the mapping relationship between the copy and the logical data center is initialized, namely, the logical data center 1 (copy 1, copy 2, copy 3), and the logical data center 2 (copy 4, copy 5).
(2) And secondly, caching the mapping relation, wherein the mapping relation is cached in a temporary variable, and the landing information is counted.
(3) And thirdly, counting the drop log entries of each logic data center, wherein the largest dropped serial number of the logic data center is taken as the serial number of the logic data center (namely, center log serial number information), namely, the maximum value (SEQ 3: 1600) in (SEQ 1, SEQ2 and SEQ 3) is taken as the drop log entry of the logic data center 1. The maximum value (SEQ 5: 1300) of (SEQ 4, SEQ 5) is taken as the drop log entry for logical data center 2.
(4) Fourth, all entries of the logical data center (logical data center 1[ SEQ3:1600], logical data center 2[ SEQ5:1300 ]) are counted, and the smallest log entry (SEQ 5:1300) is selected as the largest position (namely, logical log serial number information) which can be submitted under the rule of submitting the logical data center. This location (SEQ 5: 1300) may ensure that at least one copy has been dropped at each logical data center (in this example, the corresponding journal entries were dropped on copies 1,2,3 of logical data center 1 and copy 5 of logical data center 2), and may ensure that the largest journal entry sequence number (for 1200, although there is a copy drop at both logical data centers, but less than 1300; for 1400, there is no copy drop in both copies of logical data center 2) that satisfies the above conditions. SEQ5:1300 is the maximum available commit position under the logical data center commit rule.
(5) Fifth, counting the median of all copies (SEQ 1:1400, SEQ2:1500, SEQ3:1600, SEQ4:1200, SEQ5: 1300) that the log of more than half copies has fallen down, wherein the position represents the maximum position of the log of the copies that the log of more than half copies has fallen down, and the position is the maximum position that can be submitted (namely physical log sequence number information) under the RAFT protocol. And then taking the smaller value (SEQ 5:1300) of the position (SEQ 5:1300) which can be submitted to the maximum according to the logic data center and the position (SEQ 1:1400) which can be submitted to the maximum under the RAFT protocol in the (4), wherein the position is the position (namely target log sequence number information) which can be submitted to the maximum according to the logic data center and the RAFT protocol, and the position is the position of the multi-copy cluster which meets the logic data center submitting rule and the submitting rule of the RAFT protocol. The LEADER then commits the log entries according to a commit sequence number of 1300 or less.
(6) And sixthly, after the LEADER is submitted, the next round of submission flow can be started, the dropped disc information of each copy is obtained again, the log entry is submitted forward continuously, and the position of each submission is increased continuously.
According to the technical scheme provided by the embodiment of the invention, by introducing the concept of the logic data center, most of submitting rules of the RAFT protocol are improved, so that the requirement of RPO=0 can be met in a scene that all copies in a physical data center cannot be recovered when a certain physical data center fails in a multi-place physical data center configuration scene.
Example IV
Fig. 4 is a schematic structural diagram of a log entry submission device according to a fourth embodiment of the present invention, where the present embodiment is applicable to a case of submitting log entries in an application scenario of a multi-copy cluster. As shown in fig. 4, the specific structure of the device includes:
a first determining module 21, configured to determine a copy mapping relationship when submitting the log entry to be submitted in each round, where the copy mapping relationship indicates a logical data center to which each copy node in the multi-copy cluster belongs;
a second determining module 22, configured to determine log sequence number information corresponding to each copy node, where the log sequence number information indicates a maximum sequence number of a log entry stored in the corresponding copy node;
A third determining module 23, configured to determine target log sequence number information according to the copy mapping relationship and the plurality of log sequence number information, so that at least one copy node of each logical data center stores log entries to be submitted;
a submitting module 24, configured to submit the journal entries to be submitted according to the target journal serial number information;
the logical data centers are data centers to which logically set duplicate nodes belong, the number of the logical data centers is multiple, and each logical data center comprises one or more duplicate nodes.
The log entry submitting device provided by the embodiment firstly determines a copy mapping relation when submitting the log entry to be submitted through a first determining module in each round, wherein the copy mapping relation indicates a logic data center to which each copy node in the multi-copy cluster belongs; then determining the corresponding log sequence number information of each copy node through a second determining module, wherein the log sequence number information indicates the maximum sequence number of the log entries stored in the corresponding copy node; then determining target log sequence number information according to the copy mapping relation and the plurality of log sequence number information through a third determining module so that at least one copy node of each logic data center stores log entries to be submitted; finally, the submitting module 24 is configured to submit the journal entries to be submitted according to the target journal serial number information.
Further, the third determining module 23 is specifically configured to:
according to the copy mapping relation and the plurality of log sequence number information, determining central log sequence number information corresponding to each logic data center;
determining logic log sequence number information according to the central log sequence number information corresponding to each logic data center;
the method comprises the steps of determining the median of serial numbers of log entries indicated by a plurality of pieces of log serial number information as physical log serial number information;
determining target log sequence number information according to the logic log sequence number information and the physical log sequence number information;
the physical log serial number information indicates serial numbers meeting log entry submitting conditions corresponding to physical data centers, the physical data centers are data centers to which the replica nodes belong in set practical application, the number of the physical data centers is one or more, and each physical data center comprises one or more replica nodes; the journal entry commit condition is that more than half of all the replica nodes included in all the physical data centers can store journal entries to be committed.
Further, the third determining module 23 is specifically configured to:
for each logic data center, determining each copy node included in the logic data center according to the copy mapping relation;
Determining log sequence number information corresponding to each copy node included in the logic data center in the plurality of log sequence number information;
and according to the determined log sequence number information, determining central log sequence number information, wherein the central log sequence number information indicates the maximum sequence number in the sequence numbers indicated by the log sequence number information.
Further, the third determining module 23 is specifically configured to:
and determining logic log sequence number information according to the central log sequence number information corresponding to each logic data center, wherein the logic log sequence number information indicates the minimum sequence number in the sequence numbers indicated by the central log sequence number information.
Further, the third determining module 23 is specifically configured to:
and determining target log sequence number information according to the logic log sequence number information and the physical log sequence number information, wherein the target log sequence number information indicates the minimum sequence number of the sequence numbers indicated by the logic log sequence number information and the sequence numbers indicated by the physical log sequence number information.
Further, the submitting module 24 is specifically configured to:
and submitting the journal entries to be submitted with the target submitting sequence number, wherein the target submitting sequence number does not exceed the sequence number indicated by the target journal sequence number information.
Further, the device further comprises:
and the serial number updating module is used for updating the log serial number information corresponding to each copy node when the corresponding copy node receives the log entry each time.
The log entry submitting device provided by the embodiment of the invention can execute the log entry submitting method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the executing method.
Example five
Fig. 5 shows a schematic diagram of the structure of an electronic device that may be used to implement an embodiment of the invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 5, the electronic device 10 includes at least one processor 11, and a memory, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc., communicatively connected to the at least one processor 11, in which the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 may also be stored. The processor 11, the ROM 12 and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.
Various components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, etc.; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as the log entry commit method.
In some embodiments, the journal entry submission method may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more of the steps of the log entry commit method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the log entry commit method in any other suitable manner (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.

Claims (10)

1. A log entry commit method, the method performed by a leader node in a multi-copy cluster, the method comprising:
determining a copy mapping relation when submitting the journal entries to be submitted in each round, wherein the copy mapping relation indicates a logic data center to which each copy node in the multi-copy cluster belongs;
determining log sequence number information corresponding to each copy node, wherein the log sequence number information indicates the maximum sequence number of a log entry stored in the corresponding copy node;
Determining target log sequence number information according to the copy mapping relation and the plurality of log sequence number information, so that at least one copy node of each logical data center stores the log entries to be submitted;
submitting the journal entries to be submitted according to the target journal serial number information;
the logical data centers are data centers to which logically set duplicate nodes belong, the number of the logical data centers is multiple, and each logical data center comprises one or more duplicate nodes.
2. The method of claim 1, wherein determining target log sequence number information based on the copy mapping relationship and the plurality of log sequence number information comprises:
according to the copy mapping relation and the log sequence number information, determining central log sequence number information corresponding to each logic data center;
determining logic log sequence number information according to the central log sequence number information corresponding to each logic data center;
determining the median of the serial numbers of the log entries indicated by the plurality of log serial number information as physical log serial number information;
determining the target log sequence number information according to the logic log sequence number information and the physical log sequence number information;
The physical log serial number information indicates serial numbers meeting log entry submitting conditions corresponding to physical data centers, wherein the physical data centers are data centers to which copy nodes in set practical application belong, the number of the physical data centers is one or more, and each physical data center comprises one or more copy nodes; and the journal entry submitting condition is that more than half of all the duplicate nodes included in the physical data center can store the journal entries to be submitted.
3. The method of claim 2, wherein determining central log sequence number information corresponding to each logical data center based on the copy mapping relationship and the plurality of log sequence number information comprises:
for each logic data center, determining each copy node included in the logic data center according to the copy mapping relation;
determining log sequence number information corresponding to each copy node included in the logic data center in the plurality of log sequence number information;
and determining the central log sequence number information according to the determined log sequence number information, wherein the central log sequence number information indicates the maximum sequence number in the sequence numbers indicated by the log sequence number information.
4. The method of claim 2, wherein determining logical log sequence number information based on the central log sequence number information corresponding to each logical data center comprises:
and determining logic log sequence number information according to the central log sequence number information corresponding to each logic data center, wherein the logic log sequence number information indicates the minimum sequence number in the sequence numbers indicated by the central log sequence number information.
5. The method of claim 2, wherein determining the target log sequence number information from the logical log sequence number information and the physical log sequence number information comprises:
and determining the target log sequence number information according to the logic log sequence number information and the physical log sequence number information, wherein the target log sequence number information indicates the minimum sequence number of the sequence numbers indicated by the logic log sequence number information and the sequence numbers indicated by the physical log sequence number information.
6. The method of claim 1, wherein submitting the log entry to be submitted in accordance with the target log sequence number information comprises:
and submitting the journal entry to be submitted with a target submitting sequence number, wherein the target submitting sequence number does not exceed the sequence number indicated by the target journal sequence number information.
7. The method as recited in claim 1, further comprising:
and for the log sequence number information corresponding to each copy node, updating the log sequence number information when the corresponding copy node receives the log entry each time.
8. A log entry submission apparatus, comprising:
the first determining module is used for determining a copy mapping relation when the journal entries to be submitted are submitted in each round, wherein the copy mapping relation indicates a logic data center to which each copy node in the multi-copy cluster belongs;
the second determining module is used for determining the log sequence number information corresponding to each copy node, wherein the log sequence number information indicates the maximum sequence number of the log entries stored in the corresponding copy node;
the third determining module is used for determining target log sequence number information according to the copy mapping relation and the log sequence number information so that at least one copy node of each logic data center stores the log entries to be submitted;
the submitting module is used for submitting the journal entries to be submitted according to the target journal serial number information;
the logical data centers are data centers to which logically set duplicate nodes belong, the number of the logical data centers is multiple, and each logical data center comprises one or more duplicate nodes.
9. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any of claims 1-7.
CN202310511899.2A 2023-05-08 2023-05-08 Log entry submitting method and device, electronic equipment and storage medium Pending CN116521085A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310511899.2A CN116521085A (en) 2023-05-08 2023-05-08 Log entry submitting method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310511899.2A CN116521085A (en) 2023-05-08 2023-05-08 Log entry submitting method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN116521085A true CN116521085A (en) 2023-08-01

Family

ID=87391873

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310511899.2A Pending CN116521085A (en) 2023-05-08 2023-05-08 Log entry submitting method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN116521085A (en)

Similar Documents

Publication Publication Date Title
US7693882B2 (en) Replicating data across the nodes in a cluster environment
EP3791285A1 (en) Multi-master architectures for distributed databases
US20180165343A1 (en) Quorum-based replication of data records
CN111339191B (en) Data storage method, device, equipment and medium of block chain
CN113364877B (en) Data processing method, device, electronic equipment and medium
CN109726211B (en) Distributed time sequence database
US11522966B2 (en) Methods, devices and systems for non-disruptive upgrades to a replicated state machine in a distributed computing environment
CN111404737B (en) Disaster recovery processing method and related device
CN113347238A (en) Message partitioning method, system, device and storage medium based on block chain
CN112598510A (en) Resource data processing method and device
CN116540938A (en) Data reading method, device, distributed storage system, equipment and storage medium
CN116521085A (en) Log entry submitting method and device, electronic equipment and storage medium
CN117082046A (en) Data uploading method, device, equipment and storage medium
CN114518848B (en) Method, device, equipment and medium for processing stored data
CN112799879B (en) Fault processing method, device, equipment and storage medium of node
CN114721869A (en) Account balance processing method and system
CN117331944A (en) Database table generation method, device, equipment and storage medium
CN118227579A (en) Cross-domain cluster data processing method, device, equipment and medium
CN116414625A (en) Log management method, device, equipment and storage medium
CN116881229A (en) Database exercise environment construction method, device, equipment and storage medium
CN117370354A (en) Snapshot and query method and device of metadata tree and electronic equipment
CN118295982A (en) REDO log replay method, REDO log replay device, REDO log replay equipment and storage medium
CN117539863A (en) Data management method, device and system, electronic equipment and storage medium
CN115118638A (en) Method, device and medium for monitoring back-end network card
CN115858104A (en) Method, device and equipment for acquiring snapshot of transaction commit sequence number and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination