CN116431294A - Long transaction screening method and device of distributed database and electronic equipment - Google Patents

Long transaction screening method and device of distributed database and electronic equipment Download PDF

Info

Publication number
CN116431294A
CN116431294A CN202211740964.0A CN202211740964A CN116431294A CN 116431294 A CN116431294 A CN 116431294A CN 202211740964 A CN202211740964 A CN 202211740964A CN 116431294 A CN116431294 A CN 116431294A
Authority
CN
China
Prior art keywords
transaction
time
active
active transaction
computing node
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211740964.0A
Other languages
Chinese (zh)
Inventor
李小航
任济双
徐文锋
张鹏
付裕
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jinzhuan Xinke Co Ltd
Original Assignee
Jinzhuan Xinke Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jinzhuan Xinke Co Ltd filed Critical Jinzhuan Xinke Co Ltd
Priority to CN202211740964.0A priority Critical patent/CN116431294A/en
Publication of CN116431294A publication Critical patent/CN116431294A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/466—Transaction processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G06F16/217—Database tuning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/46—Indexing scheme relating to G06F9/46
    • G06F2209/462—Lookup
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The invention discloses a long transaction screening method and device of a distributed database and electronic equipment. The method comprises the following steps: controlling a computing node to screen a long transaction according to the application time of all active transaction IDs in a first list every first time, wherein the first list is all active transaction IDs cached by a global transaction management node; or controlling the global transaction management node to check all active transaction IDs in the first list once every second time period, and screening out long transactions according to a target screening mechanism. The invention solves the technical problem of large accumulation of long transactions in the distributed database.

Description

Long transaction screening method and device of distributed database and electronic equipment
Technical Field
The present invention relates to the field of computers, and in particular, to a method and an apparatus for screening long transactions in a distributed database, and an electronic device.
Background
In a distributed database, because long transaction processing occupies a large amount of time, the database system is low in efficiency and very slow in response due to accumulation of a large amount of long transactions, even if an error condition occurs due to overtime of system response, real-time synchronization and user experience of database data are seriously affected, and therefore, how to identify and timely process long transactions within a certain time is very important. Most of the distributed databases at present have imperfect detection mechanisms for long transactions, low efficiency or too single detection mechanism. Under a single detection mechanism, if the detection mechanism fails, a large amount of long transactions in the system are accumulated, and the long transactions cannot be efficiently and accurately screened, so that the long transactions in the distributed database cannot be well processed, and the operation efficiency of the system is seriously affected.
Disclosure of Invention
The embodiment of the invention provides a long transaction screening method and device of a distributed database and electronic equipment, which can solve the technical problem of large accumulation of long transactions in the distributed database.
According to an aspect of an embodiment of the present invention, there is provided a long transaction screening method of a distributed database, including: controlling a computing node to screen a long transaction once according to the application time of all active transaction IDs in a first list every first time, wherein the first list is all active transaction IDs cached by a global transaction management node; or controlling the global transaction management node to check all active transaction IDs in the first list once every second time interval, and screening out the long transaction according to a target screening mechanism.
According to another aspect of the embodiment of the present invention, there is provided a long transaction screening apparatus of a distributed database, including: the first control module is used for controlling the computing node to screen a long transaction according to the application time of all active transaction IDs in a first list every interval of a first time length, wherein the first list is all active transaction IDs cached by the global transaction management node; or controlling the global transaction management node to check all active transaction IDs in the first list once every second time interval, and screening out the long transaction according to a target screening mechanism.
As an optional example, the first control module includes: the first control unit is used for controlling the computing node to send a query request to the global transaction management node every interval of the first time so that the global transaction management node sends relevant information of all active transaction IDs in the first list to the computing node, wherein the relevant information comprises application time; the second control unit is used for controlling the computing node to calculate the existence time of all the active transaction IDs according to the query time and the application time of all the active transaction IDs, wherein the query time is the time generated by the query request; and a third control unit, configured to control the computing node to screen a target active transaction ID from all the active transaction IDs, and determine a target transaction corresponding to the target active transaction ID as a long transaction, where a time period of existence of the target active transaction ID is longer than a preset time period.
As an alternative example, the second control unit includes: the processing subunit is configured to use the all active transaction IDs as current active transaction IDs, and perform the following operations on the current active transaction IDs: and controlling the computing node to calculate the time difference between the inquiring time and the applying time of the current active transaction ID to obtain the existence duration of the current active transaction ID.
As an alternative example, the above apparatus further includes: and the second control module is used for controlling the computing node to send a first release request to the global transaction management node under the condition that the target active transaction ID does not exist in a second list after the long transaction is screened out, so that the global transaction management node releases the target active transaction ID, and sending a release success signal to the computing node after the release is successful, wherein the second list is all active transaction IDs cached by the computing node.
As an alternative example, the above apparatus further includes: the inquiring module is used for inquiring the target active transaction ID in the submitted transaction rollback record under the condition that the target active transaction ID exists in the second list after the long transaction is screened out; and the third control module is used for controlling the computing node to send a second release request to the global transaction management node under the condition that the target active transaction ID is submitted and rollback is successful, so that the global transaction management node releases the target active transaction ID, and sending a release success signal to the computing node after the release is successful.
As an alternative example, the above apparatus further includes: and the first alarm module is used for sending out first alarm information when the target active transaction ID is submitted and the rollback fails after the target active transaction ID is queried in the rollback record of the submitted transaction, wherein the first alarm information is used for prompting a user that the target active transaction ID is submitted and the rollback fails.
As an alternative example, the above apparatus further includes: and the second alarm module is used for sending out second alarm information when the target active transaction ID does not exist in the submitted transaction rollback record after the target active transaction ID is queried in the submitted transaction rollback record, wherein the second alarm information is used for prompting a user that the target active transaction ID is abnormal.
According to yet another aspect of the embodiments of the present invention, there is also provided a storage medium having a computer program stored therein, wherein the computer program when executed by a processor performs the long transaction screening method of the above-described distributed database.
According to yet another aspect of an embodiment of the present invention, there is also provided an electronic device including a memory, in which a computer program is stored, and a processor configured to execute the long transaction screening method of the distributed database described above by the computer program.
In the embodiment of the invention, each interval of a first time length of a control computing node is adopted, and a long transaction is screened according to the application time of all active transaction IDs in a first list, wherein the first list is all active transaction IDs cached by a global transaction management node; or controlling the global transaction management node to check all active transaction IDs in the first list once every second time interval, and screening out the long transaction according to a target screening mechanism.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiments of the invention and together with the description serve to explain the invention and do not constitute a limitation on the invention. In the drawings:
FIG. 1 is a flow chart of an alternative long transaction screening method of a distributed database according to an embodiment of the present invention;
FIG. 2 is an overall architecture diagram of an alternative long transaction screening method for a distributed database according to an embodiment of the present invention;
FIG. 3 is a diagram of a GTM reverse check architecture of an alternative long transaction screening method for a distributed database according to an embodiment of the present invention;
FIG. 4 is a diagram of a CN self-checking architecture of an alternative long transaction screening method of a distributed database according to an embodiment of the invention;
FIG. 5 is a schematic diagram of an alternative long transaction screening device of a distributed database according to an embodiment of the present invention;
fig. 6 is a schematic diagram of an alternative electronic device according to an embodiment of the invention.
Detailed Description
In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
According to a first aspect of an embodiment of the present invention, there is provided a long transaction screening method of a distributed database, optionally, as shown in fig. 1, the method includes:
s102, controlling a computing node to screen a long transaction once according to the application time of all active transaction IDs in a first list every interval of a first time length, wherein the first list is all active transaction IDs cached by a global transaction management node; or (b)
And controlling the global transaction management node to check all active transaction IDs in the first list once every second time period, and screening out long transactions according to a target screening mechanism.
Optionally, in this embodiment, in the distributed database, the active transaction ID (GTID) is generally managed by a global transaction management node (Global Transaction Manager Node, abbreviated as GTM), when an active transaction is generated, the computing node CN applies the GTID to the global transaction management node GTM, and when the active transaction is finished, the computing node CN notifies the global transaction management node GTM to release the GTID. The GTID is unique for each transaction. Therefore, the global transaction management node GTM maintains all the current GTIDs in the distributed database system, and the atomicity and consistency of the distributed database transactions are realized through unified management.
Optionally, in this embodiment, the overall architecture diagram is as shown in fig. 2, and the client: the business application layer is responsible for issuing SQL sentences to the computation node CN; the computing node CN: and analyzing the SQL statement issued by the client, and initiating a processing request about the GTID (gateway transport protocol) to the global transaction management node GTM, wherein the processing request comprises inquiry, application, GTID releasing and the like. After receiving the processing result of the global transaction management node GTM about the GTID, the computing node CN modifies the SQL statement issued by the client and issues the SQL statement to a plurality of data node DB with interactive relation with the client, and the data node DB executes the modified SQL statement and returns the result to the computing node CN. The computation node CN generally interacts with a plurality of data node DB, and decides which data node DB to send the SQL sentence to execute according to the distribution rule; global transaction management node GTM: the method is responsible for managing global transactions in distributed data and mainly comprises the steps of allocating GTIDs for the transactions, maintaining an active GTID list, releasing the GTIDs after the transactions are finished and the like. The GTM cluster is usually composed of a GTM host and a plurality of GTM standby, links can be built between the host and the standby, data synchronization is guaranteed, when the host fails, the standby can replace the host to perform normal functions, and normal operation of the whole distributed database function is guaranteed, so that the high-availability cluster architecture has a good failure handling mechanism and disaster recovery capability. Data node DB: the main function is to execute and return results to the SQL statement issued by the compute node CN.
Optionally, in this embodiment, the long transaction may be screened by a method of self-checking by the computing node CN, or by a method of reverse checking by the global transaction management node GTM. The computing node CN self-checks: the computing node CN timing (may be screened once for a first time length every interval, and the first time length may be 5s and 10 s) screens long transactions according to a first list of all active transaction IDs cached by the global transaction management node GTM. Global transaction management node GTM review: as shown in fig. 3, the global transaction management node GTM checks the first list of all active transaction IDs in its own cache at regular time (may be screened once for every second duration, which may be 5s, 10 s), screens long transactions according to the target screening mechanism, and sends a message to the computing node CN after screening the long transactions. After receiving the message about the long transaction sent by the global transaction management node GTM, the computing node CN replies to the global transaction management node GTM.
Optionally, in this embodiment, when the distributed database system fails, the computing node CN may trigger immediately the global transaction management node GTM to check reversely when the computing node CN cannot actively initiate a request for querying the GTID to the global transaction management node GTM, and return a result to the computing node CN after screening out a long transaction by using its own target screening mechanism. Likewise, when the target screening mechanism of the global transaction management node GTM itself fails due to system reasons, the computing node CN may actively initiate a query request of the active GTID, and the global transaction management node GTM only needs to answer back, where the computing node CN may verify the returned result. Therefore, when any one of the two parties fails, smooth progress of the long transaction screening can be ensured, and the execution efficiency of the long transaction can be effectively improved.
Optionally, in this embodiment, by using a bidirectional checking method that combines self-checking of a computing node and inverse checking of a global transaction management node, a purpose of quickly and efficiently selecting long transactions in a distributed database and processing the long transactions in a short time is achieved, so that a technical problem of massive accumulation of the long transactions in the distributed database is solved.
As an optional example, controlling the computing node to filter a long transaction according to the application moments of all active transaction IDs in the first list every interval of the first duration includes:
the method comprises the steps that a computing node is controlled to send a query request to a global transaction management node every first time interval, so that the global transaction management node sends relevant information of all active transaction IDs in a first list to the computing node, wherein the relevant information comprises application time;
the control computing node calculates the existence time of all the active transaction IDs according to the query time and the application time of all the active transaction IDs, wherein the query time is the time of generating the query request;
and the control computing node screens out target active transaction IDs from all the active transaction IDs, and determines target transactions corresponding to the target active transaction IDs as long transactions, wherein the existence time of the target active transaction IDs is longer than the preset time.
Optionally, in this embodiment, as shown in fig. 4, 1, the computing node CN actively initiates a query request to the global transaction management node GTM for querying all active GTIDs at present once every first time interval. 2. After receiving the query request of the computing node CN, the global transaction management node GTM distributes the query message to the corresponding processing thread for processing, and after the query message is completed, the related information of all the active GTIDs in the first list is sent to the computing node CN, wherein the related information comprises the application time of the active GTIDs. 3. The computing node CN computes the existence time of each active GTID according to the application time of each active GTID in the first list and the time generated by the query request, determines the target active GTID with the existence time longer than the preset time as a long transaction, and finally screens to obtain the long transaction. The preset time period can be 10 minutes or 15 minutes and can be set by a user.
As an optional example, the controlling the computing node to calculate the existence duration of all active transaction IDs according to the query time and the application time of all active transaction IDs includes:
taking all the active transaction IDs as current active transaction IDs, and executing the following operations on the current active transaction IDs:
and the control computing node calculates the time difference between the inquiry time and the application time of the current active transaction ID to obtain the existence time of the current active transaction ID.
Optionally, in this embodiment, the duration of existence of each active GTID is calculated, taking the current active GTID as an example, where the application time of the current active GTID is 18:00:00, the moment of query request generation is 18:12:30, calculating the time difference between the inquiry time and the application time to be 12 minutes and 30 seconds, namely the existing time length of the current active GTID to be 12 minutes and 30 seconds.
As an alternative example, after screening out the long transaction, the method further comprises:
and under the condition that the target active transaction ID does not exist in the second list, the control computing node sends a first release request to the global transaction management node so that the global transaction management node releases the target active transaction ID, and after the release is successful, a release success signal is sent to the computing node, wherein the second list is all active transaction IDs cached by the computing node.
Optionally, in this embodiment, the computing node CN maintains a second list of active GTID lists, mainly for comparing the first list obtained from the global transaction management node GTM. After the computing node CN obtains the target active GTID of the long transaction through self-checking and global transaction management node GTM back-checking, searching the target active GTID in a second list, if the target active GTID is not in the second list, the computing node CN sends a first release request to the global transaction management node GTM, notifying the global transaction management node GTM to release the target active GTID, and simultaneously, the global transaction management node GTM returns a processing result to the computing node CN.
As an alternative example, after screening out the long transaction, the method further comprises:
querying the committed transaction rollback record for the target active transaction ID if the target active transaction ID exists in the second list;
and under the condition that the target active transaction ID is submitted and rollback is successful, the control computing node sends a second release request to the global transaction management node so that the global transaction management node releases the target active transaction ID, and after the release is successful, a release success signal is sent to the computing node.
Alternatively, in the present embodiment, if this target active transaction ID is present in the second list, the committed transaction rollback record is queried. Rollback refers to the act of reverting to the most recent correct version when a program or data is corrupted. By the rollback mechanism, the availability of the system can be ensured when the database system fails. If the long transaction has committed and the rollback is successful, the compute node CN notifies the global transaction management node GTM to release the target active transaction ID. And the global transaction management node GTM returns a processing result to the computing node CN.
As an alternative example, after querying the committed transaction rollback record for the target active transaction ID, the method further comprises:
and under the condition that the target active transaction ID is submitted and the rollback fails, sending out first alarm information, wherein the first alarm information is used for prompting the user that the target active transaction ID is submitted and the rollback fails.
Optionally, in this embodiment, if the long transaction has been submitted and the rollback fails, a first alarm message is initiated to prompt the user that the target active transaction ID has been submitted but the rollback fails, and the human intervention is processed.
As an alternative example, after querying the committed transaction rollback record for the target active transaction ID, the method further comprises:
and under the condition that the target active transaction ID does not exist in the submitted transaction rollback record, sending out second alarm information, wherein the second alarm information is used for prompting the user that the target active transaction ID is abnormal.
Optionally, in this embodiment, if the long transaction is not found in the committed transaction rollback record, a second alarm message is initiated to prompt the user that the target active transaction ID is abnormal, and notify the user that a manual intervention is required to check whether the transaction is abnormal.
It should be noted that, for simplicity of description, the foregoing method embodiments are all described as a series of acts, but it should be understood by those skilled in the art that the present invention is not limited by the order of acts described, as some steps may be performed in other orders or concurrently in accordance with the present invention. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily required for the present invention.
According to another aspect of the embodiments of the present application, there is further provided a long transaction screening apparatus of a distributed database, as shown in fig. 5, including:
the first control module 502 is configured to control the computing node to screen a long transaction according to the application time of all active transaction IDs in a first list every interval of a first duration, where the first list is all active transaction IDs cached by the global transaction management node; or (b)
And controlling the global transaction management node to check all active transaction IDs in the first list once every second time period, and screening out long transactions according to a target screening mechanism.
Optionally, in this embodiment, in the distributed database, the active transaction ID (GTID) is generally managed by a global transaction management node (Global Transaction Manager Node, abbreviated as GTM), when an active transaction is generated, the computing node CN applies the GTID to the global transaction management node GTM, and when the active transaction is finished, the computing node CN notifies the global transaction management node GTM to release the GTID. The GTID is unique for each transaction. Therefore, the global transaction management node GTM maintains all the current GTIDs in the distributed database system, and the atomicity and consistency of the distributed database transactions are realized through unified management.
Optionally, in this embodiment, the overall architecture diagram is as shown in fig. 2, and the client: the business application layer is responsible for issuing SQL sentences to the computation node CN; the computing node CN: and analyzing the SQL statement issued by the client, and initiating a processing request about the GTID (gateway transport protocol) to the global transaction management node GTM, wherein the processing request comprises inquiry, application, GTID releasing and the like. After receiving the processing result of the global transaction management node GTM about the GTID, the computing node CN modifies the SQL statement issued by the client and issues the SQL statement to a plurality of data node DB with interactive relation with the client, and the data node DB executes the modified SQL statement and returns the result to the computing node CN. The computation node CN generally interacts with a plurality of data node DB, and decides which data node DB to send the SQL sentence to execute according to the distribution rule; global transaction management node GTM: the method is responsible for managing global transactions in distributed data and mainly comprises the steps of allocating GTIDs for the transactions, maintaining an active GTID list, releasing the GTIDs after the transactions are finished and the like. The GTM cluster is usually composed of a GTM host and a plurality of GTM standby, links can be built between the host and the standby, data synchronization is guaranteed, when the host fails, the standby can replace the host to perform normal functions, and normal operation of the whole distributed database function is guaranteed, so that the high-availability cluster architecture has a good failure handling mechanism and disaster recovery capability. Data node DB: the main function is to execute and return results to the SQL statement issued by the compute node CN.
Optionally, in this embodiment, the long transaction may be screened by a method of self-checking by the computing node CN, or by a method of reverse checking by the global transaction management node GTM. The computing node CN self-checks: the computing node CN timing (may be screened once for a first time length every interval, and the first time length may be 5s and 10 s) screens long transactions according to a first list of all active transaction IDs cached by the global transaction management node GTM. Global transaction management node GTM review: as shown in fig. 3, the global transaction management node GTM checks the first list of all active transaction IDs in its own cache at regular time (may be screened once for every second duration, which may be 5s, 10 s), screens long transactions according to the target screening mechanism, and sends a message to the computing node CN after screening the long transactions. After receiving the message about the long transaction sent by the global transaction management node GTM, the computing node CN replies to the global transaction management node GTM.
Optionally, in this embodiment, when the distributed database system fails, the computing node CN may trigger immediately the global transaction management node GTM to check reversely when the computing node CN cannot actively initiate a request for querying the GTID to the global transaction management node GTM, and return a result to the computing node CN after screening out a long transaction by using its own target screening mechanism. Likewise, when the target screening mechanism of the global transaction management node GTM itself fails due to system reasons, the computing node CN may actively initiate a query request of the active GTID, and the global transaction management node GTM only needs to answer back, where the computing node CN may verify the returned result. Therefore, when any one of the two parties fails, smooth progress of the long transaction screening can be ensured, and the execution efficiency of the long transaction can be effectively improved.
Optionally, in this embodiment, by using a bidirectional checking method that combines self-checking of a computing node and inverse checking of a global transaction management node, a purpose of quickly and efficiently selecting long transactions in a distributed database and processing the long transactions in a short time is achieved, so that a technical problem of massive accumulation of the long transactions in the distributed database is solved.
As an alternative example, the first control module includes:
the first control unit is used for controlling the computing node to send a query request to the global transaction management node every a first time length, so that the global transaction management node sends relevant information of all active transaction IDs in the first list to the computing node, wherein the relevant information comprises application time;
the second control unit is used for controlling the computing node to calculate the existence duration of all the active transaction IDs according to the query time and the application time of all the active transaction IDs, wherein the query time is the time generated by the query request;
and the third control unit is used for controlling the computing node to screen out target active transaction IDs from all active transaction IDs, and determining target transactions corresponding to the target active transaction IDs as long transactions, wherein the existence time of the target active transaction IDs is longer than the preset time.
Optionally, in this embodiment, as shown in fig. 4, 1, the computing node CN actively initiates a query request to the global transaction management node GTM for querying all active GTIDs at present once every first time interval. 2. After receiving the query request of the computing node CN, the global transaction management node GTM distributes the query message to the corresponding processing thread for processing, and after the query message is completed, the related information of all the active GTIDs in the first list is sent to the computing node CN, wherein the related information comprises the application time of the active GTIDs. 3. The computing node CN computes the existence time of each active GTID according to the application time of each active GTID in the first list and the time generated by the query request, determines the target active GTID with the existence time longer than the preset time as a long transaction, and finally screens to obtain the long transaction. The preset time period can be 10 minutes or 15 minutes and can be set by a user.
As an alternative example, the second control unit includes:
the processing subunit is configured to take all the active transaction IDs as current active transaction IDs, and perform the following operations on the current active transaction IDs:
and the control computing node calculates the time difference between the inquiry time and the application time of the current active transaction ID to obtain the existence time of the current active transaction ID.
Optionally, in this embodiment, the duration of existence of each active GTID is calculated, taking the current active GTID as an example, where the application time of the current active GTID is 18:00:00, the moment of query request generation is 18:12:30, calculating the time difference between the inquiry time and the application time to be 12 minutes and 30 seconds, namely the existing time length of the current active GTID to be 12 minutes and 30 seconds.
As an alternative example, the above apparatus further includes:
and the second control module is used for controlling the computing node to send a first release request to the global transaction management node under the condition that the target active transaction ID does not exist in the second list after the long transaction is screened, so that the global transaction management node releases the target active transaction ID, and sending a release success signal to the computing node after the release is successful, wherein the second list is all the active transaction IDs cached by the computing node.
Optionally, in this embodiment, the computing node CN maintains a second list of active GTID lists, mainly for comparing the first list obtained from the global transaction management node GTM. After the computing node CN obtains the target active GTID of the long transaction through self-checking and global transaction management node GTM back-checking, searching the target active GTID in a second list, if the target active GTID is not in the second list, the computing node CN sends a first release request to the global transaction management node GTM, notifying the global transaction management node GTM to release the target active GTID, and simultaneously, the global transaction management node GTM returns a processing result to the computing node CN.
As an alternative example, the above apparatus further includes:
the inquiring module is used for inquiring the target active transaction ID in the submitted transaction rollback record under the condition that the target active transaction ID exists in the second list after the long transaction is screened;
and the third control module is used for controlling the computing node to send a second release request to the global transaction management node under the condition that the target active transaction ID is submitted and rollback is successful, so that the global transaction management node releases the target active transaction ID, and sending a release success signal to the computing node after the release is successful.
Alternatively, in the present embodiment, if this target active transaction ID is present in the second list, the committed transaction rollback record is queried. Rollback refers to the act of reverting to the most recent correct version when a program or data is corrupted. By the rollback mechanism, the availability of the system can be ensured when the database system fails. If the long transaction has committed and the rollback is successful, the compute node CN notifies the global transaction management node GTM to release the target active transaction ID. And the global transaction management node GTM returns a processing result to the computing node CN.
As an alternative example, the above apparatus further includes:
and the first alarm module is used for sending out first alarm information when the target active transaction ID is submitted and the rollback fails after the target active transaction ID is queried in the rollback record of the submitted transaction, wherein the first alarm information is used for prompting the user that the target active transaction ID is submitted and the rollback fails.
Optionally, in this embodiment, if the long transaction has been submitted and the rollback fails, a first alarm message is initiated to prompt the user that the target active transaction ID has been submitted but the rollback fails, and the human intervention is processed.
As an alternative example, the above apparatus further includes:
and the second alarm module is used for sending out second alarm information under the condition that the target active transaction ID does not exist in the submitted transaction rollback record after the target active transaction ID is queried in the submitted transaction rollback record, wherein the second alarm information is used for prompting the user that the target active transaction ID is abnormal.
Optionally, in this embodiment, if the long transaction is not found in the committed transaction rollback record, a second alarm message is initiated to prompt the user that the target active transaction ID is abnormal, and notify the user that a manual intervention is required to check whether the transaction is abnormal.
For other examples of this embodiment, please refer to the above examples, and are not described herein.
Fig. 6 is a block diagram of an alternative electronic device, according to an embodiment of the present application, including a processor 602, a communication interface 604, a memory 606, and a communication bus 608, as shown in fig. 6, wherein the processor 602, the communication interface 604, and the memory 606 communicate with each other via the communication bus 608, wherein,
a memory 606 for storing a computer program;
the processor 602, when executing the computer program stored on the memory 606, performs the following steps:
controlling a computing node to screen a long transaction according to the application time of all active transaction IDs in a first list every first time, wherein the first list is all active transaction IDs cached by a global transaction management node; or (b)
And controlling the global transaction management node to check all active transaction IDs in the first list once every second time period, and screening out long transactions according to a target screening mechanism.
Alternatively, in the present embodiment, the above-described communication bus may be a PCI (Peripheral Component Interconnect, peripheral component interconnect standard) bus, or an EISA (Extended Industry Standard Architecture ) bus, or the like. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, only one thick line is shown in fig. 6, but not only one bus or one type of bus. The communication interface is used for communication between the electronic device and other devices.
The memory may include RAM or may include non-volatile memory (non-volatile memory), such as at least one disk memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.
As an example, the memory 606 may include, but is not limited to, the first control module 502 in a long transaction screening device that includes the distributed database. In addition, other module units in the processing apparatus of the above request may be included, but are not limited to, and are not described in detail in this example.
The processor may be a general purpose processor and may include, but is not limited to: CPU (Central Processing Unit ), NP (Network Processor, network processor), etc.; but also DSP (Digital Signal Processing, digital signal processor), ASIC (Application Specific Integrated Circuit ), FPGA (Field-Programmable Gate Array, field programmable gate array) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components.
Alternatively, specific examples in this embodiment may refer to examples described in the foregoing embodiments, and this embodiment is not described herein.
It will be appreciated by those skilled in the art that the structure shown in fig. 6 is only illustrative, and the device implementing the long transaction screening method of the distributed database may be a terminal device, and the terminal device may be a smart phone (such as an Android mobile phone, an iOS mobile phone, etc.), a tablet computer, a palm computer, a mobile internet device (Mobile Internet Devices, MID), a PAD, etc. Fig. 6 is not limited to the structure of the electronic device described above. For example, the electronic device may also include more or fewer components (e.g., network interfaces, display devices, etc.) than shown in FIG. 6, or have a different configuration than shown in FIG. 6.
Those of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be implemented by a program for instructing a terminal device to execute in association with hardware, the program may be stored in a computer readable storage medium, and the storage medium may include: flash disk, ROM, RAM, magnetic or optical disk, etc.
According to yet another aspect of embodiments of the present invention, there is also provided a computer-readable storage medium having a computer program stored therein, wherein the computer program when executed by a processor performs the steps in the long transaction screening method of a distributed database as described above.
Alternatively, in this embodiment, it will be understood by those skilled in the art that all or part of the steps in the methods of the above embodiments may be performed by a program for instructing a terminal device to execute the steps, where the program may be stored in a computer readable storage medium, and the storage medium may include: flash disk, read-Only Memory (ROM), random-access Memory (Random Access Memory, RAM), magnetic or optical disk, and the like.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
The integrated units in the above embodiments may be stored in the above-described computer-readable storage medium if implemented in the form of software functional units and sold or used as separate products. Based on such understanding, the technical solution of the present invention may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, comprising several instructions for causing one or more computer devices (which may be personal computers, servers or network devices, etc.) to perform all or part of the steps of the method described in the embodiments of the present invention.
In the foregoing embodiments of the present invention, the descriptions of the embodiments are emphasized, and for a portion of this disclosure that is not described in detail in this embodiment, reference is made to the related descriptions of other embodiments.
In several embodiments provided in the present application, it should be understood that the disclosed client may be implemented in other manners. The above-described embodiments of the apparatus are merely exemplary, and the division of the units, such as the division of the units, is merely a logical function division, and may be implemented in another manner, for example, multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some interfaces, units or modules, or may be in electrical or other forms.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The foregoing is merely a preferred embodiment of the present invention and it should be noted that modifications and adaptations to those skilled in the art may be made without departing from the principles of the present invention, which are intended to be comprehended within the scope of the present invention.

Claims (10)

1. A method for long transaction screening of a distributed database, comprising:
controlling a computing node to screen a long transaction once according to the application time of all active transaction IDs in a first list every first time, wherein the first list is all active transaction IDs cached by a global transaction management node; or (b)
And controlling the global transaction management node to check all active transaction IDs in the first list once every second time interval, and screening out the long transaction according to a target screening mechanism.
2. The method of claim 1, wherein the controlling the computing node to filter one long transaction based on the application times of all active transaction IDs in the first list every first time period comprises:
controlling the computing node to send a query request to the global transaction management node every time the first time length is set, so that the global transaction management node sends relevant information of all active transaction IDs in the first list to the computing node, wherein the relevant information comprises application time;
the computing node is controlled to calculate the existence time of all the active transaction IDs according to the query time and the application time of all the active transaction IDs, wherein the query time is the time generated by the query request;
and controlling the computing node to screen out a target active transaction ID from all the active transaction IDs, and determining a target transaction corresponding to the target active transaction ID as a long transaction, wherein the existence time of the target active transaction ID is longer than a preset time.
3. The method according to claim 2, wherein the controlling the computing node to calculate the existence duration of all the active transaction IDs according to the query time and the application time of all the active transaction IDs comprises:
taking all the active transaction IDs as current active transaction IDs, and executing the following operations on the current active transaction IDs:
and controlling the computing node to calculate the time difference between the inquiring time and the applying time of the current active transaction ID to obtain the existing time length of the current active transaction ID.
4. The method of claim 1, wherein after screening out the long transaction, the method further comprises:
and under the condition that the target active transaction ID does not exist in the second list, controlling the computing node to send a first release request to the global transaction management node so that the global transaction management node releases the target active transaction ID, and sending a release success signal to the computing node after the release is successful, wherein the second list is all active transaction IDs cached by the computing node.
5. The method of claim 1, wherein after screening out the long transaction, the method further comprises:
querying a committed transaction rollback record for a target active transaction ID if the target active transaction ID exists in a second list;
and under the condition that the target active transaction ID is submitted and rollback is successful, controlling the computing node to send a second release request to the global transaction management node so that the global transaction management node releases the target active transaction ID, and after release is successful, sending a release success signal to the computing node.
6. The method of claim 4, wherein after querying the committed transaction rollback record for the target active transaction ID, the method further comprises:
and under the condition that the target active transaction ID is submitted and rollback fails, sending out first alarm information, wherein the first alarm information is used for prompting a user that the target active transaction ID is submitted and rollback fails.
7. The method of claim 4, wherein after querying the committed transaction rollback record for the target active transaction ID, the method further comprises:
and under the condition that the target active transaction ID does not exist in the submitted transaction rollback record, sending out second alarm information, wherein the second alarm information is used for prompting a user that the target active transaction ID is abnormal.
8. A long transaction screening device for a distributed database, comprising:
the first control module is used for controlling the computing node to screen a long transaction according to the application time of all active transaction IDs in a first list every interval of a first time length, wherein the first list is all active transaction IDs cached by the global transaction management node; or (b)
And controlling the global transaction management node to check all active transaction IDs in the first list once every second time interval, and screening out the long transaction according to a target screening mechanism.
9. A computer-readable storage medium, having stored thereon a computer program, characterized in that the computer program, when executed by a processor, performs the method of any of claims 1 to 7.
10. An electronic device comprising a memory and a processor, characterized in that the memory has stored therein a computer program, the processor being arranged to execute the method according to any of the claims 1 to 7 by means of the computer program.
CN202211740964.0A 2022-12-30 2022-12-30 Long transaction screening method and device of distributed database and electronic equipment Pending CN116431294A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211740964.0A CN116431294A (en) 2022-12-30 2022-12-30 Long transaction screening method and device of distributed database and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211740964.0A CN116431294A (en) 2022-12-30 2022-12-30 Long transaction screening method and device of distributed database and electronic equipment

Publications (1)

Publication Number Publication Date
CN116431294A true CN116431294A (en) 2023-07-14

Family

ID=87086110

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211740964.0A Pending CN116431294A (en) 2022-12-30 2022-12-30 Long transaction screening method and device of distributed database and electronic equipment

Country Status (1)

Country Link
CN (1) CN116431294A (en)

Similar Documents

Publication Publication Date Title
US11379461B2 (en) Multi-master architectures for distributed databases
US10007715B1 (en) Database replication
US9984140B1 (en) Lease based leader election system
US10831741B2 (en) Log-shipping data replication with early log record fetching
US10884623B2 (en) Method and apparatus for upgrading a distributed storage system
CN113268471B (en) Method, proxy connection pool, system, device and medium for processing distributed transaction
CN111338773A (en) Distributed timed task scheduling method, scheduling system and server cluster
CN106933550B (en) Global information obtaining, processing and updating method, device and system
CN108664521B (en) Method and device for providing serial number, electronic equipment and readable storage medium
CN113064744A (en) Task processing method and device, computer readable medium and electronic equipment
CN112685499A (en) Method, device and equipment for synchronizing process data of work service flow
US20200026428A1 (en) Smart auto-backup of virtual machines using a virtual proxy
CN111311254A (en) Service processing method, device and system based on block chain
CN109885382A (en) The system of cross-system distributed transaction processing method and distributing real time system
CN110941622A (en) Data processing method and device
CN112561506B (en) Live broadcast data processing method, system, equipment and medium based on virtual currency
US20240004746A1 (en) Access Consistency in High-Availability Databases
US11243979B1 (en) Asynchronous propagation of database events
CN116431294A (en) Long transaction screening method and device of distributed database and electronic equipment
CN112711466B (en) Hanging affair inspection method and device, electronic equipment and storage medium
CN115858668A (en) Distributed transaction processing method, device, electronic device and storage medium
Sun et al. Adaptive trade‐off between consistency and performance in data replication
CN106375354B (en) Data processing method and device
CN110737649A (en) Processing method, device and storage medium for transaction log of databases
CN106878369B (en) Service processing method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination