CN114880336A - Database access method, device, equipment and storage medium - Google Patents

Database access method, device, equipment and storage medium Download PDF

Info

Publication number
CN114880336A
CN114880336A CN202210486427.1A CN202210486427A CN114880336A CN 114880336 A CN114880336 A CN 114880336A CN 202210486427 A CN202210486427 A CN 202210486427A CN 114880336 A CN114880336 A CN 114880336A
Authority
CN
China
Prior art keywords
time
service request
current
time interval
delay value
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210486427.1A
Other languages
Chinese (zh)
Inventor
郝国庆
贺军
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Construction Bank Corp
Original Assignee
China Construction Bank Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Construction Bank Corp filed Critical China Construction Bank Corp
Priority to CN202210486427.1A priority Critical patent/CN114880336A/en
Publication of CN114880336A publication Critical patent/CN114880336A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating
    • G06F16/2365Ensuring data consistency and integrity
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/242Query formulation
    • G06F16/2423Interactive query statement specification based on a database schema
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28Databases characterised by their database models, e.g. relational or object models
    • G06F16/284Relational databases

Abstract

The application relates to the technical field of computers, and provides a database access method, a device, equipment and a storage medium, wherein the method acquires a service request and judges whether the service request is a query request; if the service request is a query request, acquiring a replication delay value between the current main node and the auxiliary node according to the delay acquisition time interval; comparing the replication delay value with a preset timeout delay threshold value; if the copying delay value is not greater than the preset overtime delay threshold value, acquiring the time of executing the data manipulation language of the service request at the latest time; and determining the routing mode of the service request according to the current time and the time of executing the data manipulation language last time.

Description

Database access method, device, equipment and storage medium
Technical Field
The present application relates to the field of computer technologies, and in particular, to a database access method, apparatus, device, and storage medium.
Background
A database is a "warehouse that organizes, stores, and manages data according to a data structure. Is an organized, sharable, uniformly managed collection of large amounts of data that is stored long term within a computer.
The data synchronization mode of the high-availability architecture of the database is generally two, including a data distribution technology and a read-write separation technology. The slave nodes on the data distribution technology line are used as new master nodes after the master node and the slave node are switched, bear the same amount of services, and the master nodes and the slave nodes are high-configuration servers, which can cause waste of resources; in order to overcome the traditional read-write separation mode of resource waste, the read request is simply forwarded to the slave node, and the data read-write consistency is not considered. Currently, in order to achieve data read-write consistency, the prior art specifies a specific Query Structured Query Language (SQL) route to a read node, such as SQL that is not sensitive to data read-write consistency or SQL-like route to a read node for a dead data table.
However, in the prior art, each SQL needs to be manually checked, whether the SQL has no requirement on the data read-write consistency is judged according to the service scene, or whether the SQL only queries a dead data table is judged, the database access workload is large, the intelligent degree is low, and the database access efficiency is low.
Disclosure of Invention
The application provides a database access method, a device, equipment and a storage medium, so that the technical problems that in the prior art, each SQL needs to be manually checked, whether the SQL has no requirement on the read-write consistency of data or not is judged according to a service scene, whether the SQL only queries a dead data table or not is judged, the database access workload is large, the intelligent degree is low, and the database access efficiency is low are solved.
In a first aspect, the present application provides a database access method, including:
acquiring a service request, and judging whether the service request is an inquiry request;
if the service request is a query request, acquiring a replication delay value between the current main node and the auxiliary node according to a delay acquisition time interval;
comparing the copy delay value with a preset timeout delay threshold value;
if the replication delay value is not greater than the preset timeout delay threshold value, acquiring the time of executing the data manipulation language of the service request at the latest time;
and determining the routing mode of the service request according to the current time and the time of executing the data manipulation language last time.
Here, the present application provides a database access method, when there is a query request in a database, a replication delay value between a current master node and an auxiliary node may be automatically obtained according to a delay obtaining time interval, and a time when a table related to a service request has been subjected to a Data Management Language (DML) last time, that is, a time when the DML is executed last time, may be automatically obtained when the replication delay value is not greater than a preset timeout delay threshold, so as to determine whether there is a DML operation within a forwarding time threshold, and determine whether to route the service request to the current master node or the replication node according to the current time and the time when the DML is executed last time, thereby ensuring read-write consistency under the read-write separation condition of database access, and without manually checking each SQL, and without manually determining whether there is a requirement for Data read-write consistency according to a service scenario, or judging whether the SQL only inquires the dead data table, automatically and intelligently routing the inquired SQL types to the reading node as much as possible, and improving the utilization rate of the reading node as much as possible on the premise of meeting the requirement of service reading and writing consistency, thereby improving the database access efficiency.
Optionally, before the obtaining the replication delay value between the current primary node and the secondary node according to the delay obtaining time interval, the method further includes:
acquiring a current replication delay value, a current time interval and a last replication delay value;
and determining a delay acquisition time interval according to the current replication delay value, the current time interval and the last replication delay value.
Here, in the embodiment of the present application, the delayed obtaining time interval may be automatically determined by the current replication delay value, the current time interval, and the last replication delay value, and the frequency of obtaining the delay value is automatically adjusted, so as to further improve the efficiency of database access.
Optionally, the determining a delay obtaining time interval according to the current replication delay value, the current time interval, and the last replication delay value includes:
comparing the current replica delay value and the last replica delay value;
if the current replication delay value is smaller than the last replication delay value, determining that a first delay obtaining time interval is one half of the current time interval;
if the current replication delay value is not smaller than the last replication delay value, determining a first delay obtaining time interval as the sum of the current time interval and a preset additional value;
comparing the first delayed acquisition time interval to a preset minimum time interval;
if the first delayed acquisition time interval is smaller than the preset minimum time interval, determining the delayed acquisition time interval as the preset minimum time interval;
and if the first delayed acquisition time interval is not smaller than the preset minimum time interval, determining the delayed acquisition time interval as the first delayed acquisition time interval.
When the replication delay value is reduced, the frequency of obtaining the delay value is rapidly improved due to exponential reduction of interval time, so that the latest delay value can be obtained relatively quickly, and because the replication delay value is smaller than the last replication delay value, the time threshold of the route reading node is replaced by a smaller replication delay value obtained by the current detection, so that more query statements are forwarded to the reading node; when the copy delay value is increased, the interval time is fixedly increased, the acquisition frequency of the delay value can be rapidly reduced, the acquisition of meaningless delay values is reduced, the power consumption of database access is further reduced, the workload is reduced, and the efficiency of database access is improved.
Optionally, the determining a routing manner of the service request according to the current time and the time of executing the data manipulation language last time includes:
determining a difference between the current time and the time at which the data manipulation language was last executed;
if the difference value is smaller than a preset routing node reading time threshold value, routing the service request to the current main node;
and if the difference value is not smaller than a preset routing reading node time threshold value, routing the service request to the auxiliary node.
Here, in the embodiment of the present application, it may be determined whether there is a DML operation in the current service table within the forwarding time threshold according to a difference between the current time and the time of executing the DML last time, so that a read operation is directly performed without performing an additional write operation, workload is reduced, power consumption is reduced, and efficiency of database access is further improved.
Optionally, if the difference is smaller than a preset routing read node time threshold, routing the service request to the current host node includes:
if the difference value is smaller than a preset routing node reading time threshold value, routing the service request to the current main node;
and recording the table name and the operation completion time of the service statement.
Here, in the embodiment of the present application, after the service request is routed to the current master node to implement the write operation, the operation completion time is further recorded, so that the processing of the subsequent service request is performed according to the operation completion time.
Optionally, after the comparing the copy delay value with a preset timeout delay threshold, the method further includes:
and if the replication delay value is greater than the preset timeout delay threshold value, routing the service request to the current main node.
Here, the embodiment of the present application is directed to write operations, and directly routes a service request to a current host node, thereby implementing efficient, fast, and intelligent database access operations.
Optionally, the obtaining the time of the last execution of the data manipulation language of the service request includes:
analyzing the service request and extracting the table related to the service request, acquiring the current time and recording the time of table operation completion;
recording the time when the table operation is completed in a global record variable;
and acquiring the time of the last execution of the data manipulation language of the service request according to the global record variable.
In a second aspect, the present application provides a database access apparatus, including:
the first acquisition module is used for acquiring a service request and judging whether the service request is an inquiry request or not;
a second obtaining module, configured to obtain a replication delay value between the current primary node and the secondary node according to a delay obtaining time interval if the service request is an inquiry request;
the first comparison module is used for comparing the replication delay value with a preset timeout delay threshold value;
a third obtaining module, configured to obtain a time for executing a data manipulation language of the service request at a latest time if the replication delay value is not greater than the preset timeout delay threshold;
and the first determining module is used for determining the routing mode of the service request according to the current time and the time of executing the data manipulation language last time.
Optionally, before the second obtaining module obtains the replication delay value between the current primary node and the secondary node according to the delay obtaining time interval, the apparatus further includes:
the fourth acquisition module is used for acquiring the current replication delay value, the current time interval and the last replication delay value;
and a second determining module, configured to determine a delay obtaining time interval according to the current replication delay value, the current time interval, and the last replication delay value.
Optionally, the second determining module is specifically configured to:
comparing the current replica delay value and the last replica delay value;
if the current replication delay value is smaller than the last replication delay value, determining that a first delay obtaining time interval is one half of the current time interval;
if the current replication delay value is not smaller than the last replication delay value, determining a first delay obtaining time interval as the sum of the current time interval and a preset additional value;
comparing the first delayed acquisition time interval to a preset minimum time interval;
if the first delayed acquisition time interval is smaller than the preset minimum time interval, determining the delayed acquisition time interval as the preset minimum time interval;
and if the first delayed acquisition time interval is not smaller than the preset minimum time interval, determining the delayed acquisition time interval as the first delayed acquisition time interval.
Optionally, the first determining module is specifically configured to:
determining a difference between the current time and the time at which the data manipulation language was last executed;
if the difference value is smaller than a preset routing node reading time threshold value, routing the service request to the current main node;
and if the difference value is not smaller than a preset routing reading node time threshold value, routing the service request to the auxiliary node.
Optionally, the first determining module is further specifically configured to:
if the difference value is smaller than a preset routing node reading time threshold value, routing the service request to the current main node;
and recording the table name and the operation completion time of the service statement.
Optionally, after the first comparing module compares the replica delay value with a preset timeout delay threshold, the apparatus further includes:
and the processing module is used for routing the service request to the current main node if the replication delay value is greater than the preset timeout delay threshold value.
Optionally, the third processing module is specifically configured to:
analyzing the service request and extracting the table related to the service request, acquiring the current time and recording the time of table operation completion;
recording the time when the table operation is completed in a global record variable;
and acquiring the time of the last execution of the data manipulation language of the service request according to the global record variable.
In a third aspect, the present application provides a database access device, comprising: at least one processor and memory;
the memory stores computer-executable instructions;
the at least one processor executing the computer-executable instructions stored by the memory causes the at least one processor to perform the database access method as set forth in the first aspect above and in various possible designs of the first aspect.
In a fourth aspect, the present invention provides a computer-readable storage medium having stored thereon computer-executable instructions that, when executed by a processor, implement the database access method as set forth in the first aspect and various possible designs of the first aspect.
In a fifth aspect, the present invention provides a computer program product comprising a computer program which, when executed by a processor, implements a database access method as described above in the first aspect and in various possible designs of the first aspect.
The method can automatically acquire the copy delay value between the current main node and the auxiliary node according to the delay acquisition time interval when the database has a query request, and automatically acquire the time when the table related to the service request has been subjected to DML last time under the condition that the copy delay value is not greater than the preset timeout delay threshold value, namely the time when the DML is executed last time, thereby determining whether the table has DML operation within the forwarding time threshold value or not, thereby determining whether the service request is routed to the current main node or the copy node according to the current time and the time when the DML is executed last time, ensuring the read-write consistency under the condition of read-write separation of database access without manually checking each SQL, and without manually judging whether the data read-write consistency is not required according to the service scene, or judging whether the SQL only inquires the dead data table, automatically and intelligently routing the inquired SQL types to the reading nodes as much as possible, and improving the utilization rate of the reading nodes as much as possible on the premise of meeting the requirement of business reading and writing consistency and improving the database access efficiency.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.
Fig. 1 is a schematic diagram of a database access system according to an embodiment of the present application;
fig. 2 is a schematic flowchart of a database access method according to an embodiment of the present application;
fig. 3 is a schematic flowchart of another database access method provided in an embodiment of the present application;
fig. 4 is a schematic structural diagram of a database access device according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a database access device according to an embodiment of the present application.
With the foregoing drawings in mind, certain embodiments of the disclosure have been shown and described in more detail below. These drawings and written description are not intended to limit the scope of the disclosed concepts in any way, but rather to illustrate the concepts of the disclosure to those skilled in the art by reference to specific embodiments.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
The terms "first," "second," "third," and "fourth," if any, in the description and claims of this application and the above-described figures are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
First, terms in the embodiments of the present application are explained:
copying and distributing: the database is used for realizing the data synchronization of the main node and the auxiliary node, the main copy sends the operation log to the auxiliary copy, and the auxiliary copy plays back the operation log to realize the data synchronization of the main copy and the auxiliary copy.
Read-write separation: one way of accessing the database, query class operations and update class (insert, update, delete) operations are handled by different machines.
Structured Query Language (SQL): is a special purpose programming language, a database query and programming language, used to access data and query, update and manage relational database systems.
The high-availability architecture of the database generally realizes data synchronization by using a copy distribution technology, only the master node can write in the architecture and only the slave nodes can read the architecture by adopting the architecture of one master node and a plurality of slave nodes, and the slave nodes are used as on-line hot standby and used for master-slave switching of master node faults. The online slave node is to be used as a new master node after the master-slave switching occurs, and bear the same amount of service, so the same master node is configured and is a high-configuration server, and if only the hot-standby node is used and is not used, the waste of resources is caused. Therefore, in most cases, a certain amount of read traffic is allocated to the slave node. The traditional read-write separation simply forwards the read request to the slave node, and does not consider the data read-write consistency, for example, the situation of read after write exists in the business process, and because the slave node data synchronization has delay, the problem of inconsistent data read-write can be caused by directly distributing the query SQL to the slave node. The read-write separation solution is used in the database access to improve the resource utilization rate of the standby server, and when the service has the requirement of read-write consistency, such as a service scene of reading after writing, the service can cause error report because the newly written data cannot be read.
In the current solution, a specific SQL route is specified to the read node, for example, SQL that is not sensitive to data read-write consistency, or a read-like SQL route to a dead data table is specified to the read node. However, in the above manner, each SQL needs to be manually checked, and whether the SQL has no requirement on the data read-write consistency is judged according to the service scene, or whether the SQL only queries a dead data table is judged, which is large in workload, and the SQL which is newly added needs to be checked once, which is not intelligent enough.
In order to solve the foregoing problems, embodiments of the present application provide a database access method, an apparatus, a device, and a storage medium, where the method periodically compares copy delay values between a current master node and an auxiliary node, and determines a current time and a time of executing DML last time, so as to automatically and intelligently route query-like SQL to a read node as much as possible, and improve the utilization rate of the read node as much as possible on the premise of meeting a requirement of service read-write consistency, thereby improving the efficiency of database access.
Optionally, fig. 1 is a schematic diagram of a database access system architecture provided in an embodiment of the present application. In fig. 1, the above architecture includes at least one of a receiving device 101, a processor 102, and a display device 103.
It is to be understood that the illustrated architecture of the embodiments of the present application does not constitute a specific limitation on the architecture of the database access system. In other possible embodiments of the present application, the foregoing architecture may include more or less components than those shown in the drawings, or combine some components, or split some components, or arrange different components, which may be determined according to practical application scenarios, and is not limited herein. The components shown in fig. 1 may be implemented in hardware, software, or a combination of software and hardware.
In a specific implementation process, the receiving device 101 may be an input/output interface or a communication interface.
The processor 102 may automatically and intelligently route the query SQL to the read node as much as possible by periodically comparing the copy delay values between the current master node and the auxiliary node and judging the current time and the time of executing the DML last time, so as to increase the utilization rate of the read node as much as possible and improve the efficiency of database access on the premise of satisfying the requirement of the consistency of the service read and write.
The display device 103 may be used to display the above results and the like.
The display device may also be a touch display screen for receiving user instructions while displaying the above-mentioned content to enable interaction with a user.
It should be understood that the processor may be implemented by reading instructions in the memory and executing the instructions, or may be implemented by a chip circuit.
In addition, the network architecture and the service scenario described in the embodiment of the present application are for more clearly illustrating the technical solution of the embodiment of the present application, and do not constitute a limitation to the technical solution provided in the embodiment of the present application, and it can be known by a person skilled in the art that along with the evolution of the network architecture and the appearance of a new service scenario, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.
The technical scheme of the application is described in detail by combining specific embodiments as follows:
optionally, fig. 2 is a schematic flowchart of a database access method provided in the embodiment of the present application. The execution subject of the embodiment of the present application may be the processor 102 in fig. 1, and the specific execution subject may be determined according to an actual application scenario. As shown in fig. 2, the method comprises the steps of:
s201: and acquiring a service request and judging whether the service request is an inquiry request.
Alternatively, the service request may be DML SQL or Data Definition Language (DDL) SQL.
Optionally, for DML SQL and DDL SQL, SQL statements are parsed, tables involved are extracted, the time when operations of the tables are completed is recorded, and the time is recorded in a global record variable map _ DML _ records (string, agent 64_ t).
S202: and if the service request is a query request, acquiring a replication delay value between the current main node and the auxiliary node according to the delay acquisition time interval.
Here, in the embodiment of the present application, after every delay acquisition time interval, a replication delay value between the current primary node and the secondary node is acquired
The current main node is used for executing write operation, and the auxiliary node is used for executing read operation.
Optionally, keyword matching may be performed according to the SQL statement in the service request to determine whether the service request is a query request.
Optionally, matching may be performed according to the statement in the preset query request statement library and the SQL statement of the service request to determine whether the service request is a query request.
Alternatively, the replication delay value between the current primary node and the secondary node may be obtained from historical data of the database.
Optionally, the current time is inserted into the table at the current primary node, and the time inserted by the primary node is obtained at the secondary node and compared with the current time, so as to determine the replication delay value.
Optionally, the current primary node and the secondary node of the database may be tested, and the replication delay value between the current primary node and the secondary node is obtained according to the time when the current primary node completes a transaction, the time when the secondary node receives the transaction, and the time when the secondary node completes the transaction.
S203: and comparing the copy delay value with a preset timeout delay threshold value.
It can be understood that the preset timeout delay threshold herein may be determined according to practical situations, and the embodiment of the present application does not specifically limit this.
Optionally, after comparing the replication delay value with the preset timeout delay threshold, the method includes: and if the replication delay value is greater than the preset overtime delay threshold value, routing the service request to the current main node.
In a possible implementation manner, whether a copy delay value between the current main node and the auxiliary node exceeds a preset timeout delay threshold value or not is checked, and if the copy delay value exceeds the timeout delay threshold value, the SQL is directly routed to the current main node.
Here, the embodiment of the present application is directed to write operations, and directly routes a service request to a current host node, thereby implementing efficient, fast, and intelligent database access operations.
S204: and if the copying delay value is not greater than the preset overtime delay threshold value, acquiring the time of executing the data manipulation language of the service request at the latest time.
It is to be understood that the timeout delay threshold herein may be determined according to practical situations, and the embodiment of the present application is not particularly limited thereto.
Optionally, obtaining the time of the last execution of the data manipulation language of the service request includes:
analyzing the service request and extracting the table related to the service request, acquiring the current time and recording the time of table operation completion; recording the time when the table operation is completed in a global record variable; and acquiring the time of the last execution of the data manipulation language of the service request according to the global record variable.
In one possible implementation, for query statement SELECT SQL, the SQL statement is parsed, the tables involved are extracted, and the current time is obtained; according to the table referred to at current SELECT SQL, the time when the table last made DML is obtained from the global record variable map _ DML _ records, and the time difference value is obtained by subtracting the time when the table last made DML from the current time.
S205: and determining the routing mode of the service request according to the current time and the time of executing the data manipulation language last time.
Optionally, determining a routing manner of the service request according to the current time and the time of executing the data manipulation language last time includes:
determining a difference between a current time and a time at which the data manipulation language was last executed; if the difference value is smaller than a preset routing node reading time threshold value, routing the service request to the current main node; and if the difference value is not less than the preset routing reading node time threshold value, routing the service request to the auxiliary node.
In a possible implementation manner, it is determined whether the difference is smaller than a time threshold of the route reading node, and if the difference is smaller than the time threshold, the SELECT SQL is forwarded to the current master node; if greater than the route read node time threshold, the SELECT SQL is forwarded to the secondary node.
Here, in the embodiment of the present application, it may be determined whether there is a DML operation in the current service table within the forwarding time threshold according to a difference between the current time and the time of executing the DML last time, so that a read operation is directly performed without performing an additional write operation, workload is reduced, power consumption is reduced, and efficiency of database access is further improved.
Through the method, the operation record variable of the global table is updated only after the DML is successfully executed, the DML completion time of the table is recorded in the operation record variable, when the SELECT is received, the operation record variable is inquired, the time taken by the operation record variable is responded to the DML completion time of the table, and therefore forwarding can be carried out only by judging whether the replication delay value exceeds the routing time threshold value or not. Under the service scene of the read-write consistency requirement, the query SQL is routed to the read node as much as possible so as to realize the maximum utilization of the standby machine physical resources; under the general mode, the problem that the read data is not timely caused by the copy distribution delay can be absolutely avoided.
Optionally, if the difference is smaller than a preset routing node reading time threshold, routing the service request to the current host node, including: if the difference value is smaller than a preset routing node reading time threshold value, routing the service request to the current main node; and recording the table name and the operation completion time of the business statement.
Here, in the embodiment of the present application, after the service request is routed to the current master node to implement the write operation, the operation completion time is further recorded, so that the processing of the subsequent service request is performed according to the operation completion time.
Optionally, the replication delay values of the current primary node and the secondary node may be periodically checked, and the preset route reading node time threshold may be dynamically adjusted.
The application provides a database access method, when a query request is made to a database, a copy delay value between a current main node and an auxiliary node can be automatically obtained according to a delay obtaining time interval, and the time when a table related to a service request has been subjected to DML last time, namely the time when the DML is executed last time, is automatically obtained under the condition that the copy delay value is not more than a preset overtime delay threshold value, so that whether the table has DML operation within a forwarding time threshold value or not can be determined, and therefore, whether the service request is routed to the current main node or the copy node is determined according to the current time and the time when the DML is executed last time, the read-write consistency under the condition of read-write separation of database access is ensured, each SQL does not need to be manually checked, whether the SQL has no requirement on the data read-write consistency is judged according to a service scene or not needs to be automatically and intelligently routed to the read nodes as many query types of SQL as possible only by querying the dead data table, on the premise of meeting the requirement of consistency of service reading and writing, the utilization rate of the reading nodes is improved as much as possible, and the efficiency of database access is improved.
Optionally, the obtaining frequency of the copy delay value may also be automatically adjusted in the embodiment of the present application, and accordingly, fig. 3 is a schematic flow chart of another database access method provided in the embodiment of the present application, as shown in fig. 3, the method includes:
s301: and acquiring a service request and judging whether the service request is an inquiry request.
S302: and if the service request is a query request, acquiring a current replication delay value, a current time interval and a last replication delay value.
S303: and determining a delay acquisition time interval according to the current copy delay value, the current time interval and the last copy delay value.
Optionally, determining the delay obtaining time interval according to the current replication delay value, the current time interval, and the last replication delay value includes:
comparing the current replica delay value with the last replica delay value; if the current replication delay value is smaller than the last replication delay value, determining that the first delay acquisition time interval is one half of the current time interval; if the current replication delay value is not smaller than the last replication delay value, determining that the first delay obtaining time interval is the sum of the current time interval and a preset additional value; comparing the first delayed acquisition time interval with a preset minimum time interval; if the first delayed acquisition time interval is smaller than a preset minimum time interval, determining the delayed acquisition time interval as the preset minimum time interval; and if the first delayed acquisition time interval is not smaller than the preset minimum time interval, determining the delayed acquisition time interval as the first delayed acquisition time interval.
It is understood that the preset minimum time interval may be determined according to practical situations, and the embodiment of the present application is not particularly limited thereto.
When the replication delay value is reduced, the frequency of obtaining the delay value is rapidly improved due to exponential reduction of interval time, so that the latest delay value can be obtained relatively quickly, and because the replication delay value is smaller than the last replication delay value, the time threshold of the route reading node is replaced by a smaller replication delay value obtained by the current detection, so that more query statements are forwarded to the reading node; when the copy delay value is increased, the interval time is fixedly increased, the acquisition frequency of the delay value can be rapidly reduced, the acquisition of meaningless delay values is reduced, the power consumption of database access is further reduced, the workload is reduced, and the efficiency of database access is improved.
In one possible implementation, the delay value acquisition frequency is automatically adjusted as follows: obtaining a current replication delay value, and comparing the current replication delay value with a last replication delay value; if the obtained copy delay value is smaller than the last delay value, adjusting the time interval value of the obtained copy delay value to 1/2 of the current interval value; if the copy delay value obtained at this time is larger than the last copy delay value, increasing the time interval value of the obtained copy delay value by a fixed N seconds; judging whether the new time interval value is smaller than the minimum time interval value or not, and if so, setting the time interval value as the minimum time interval value; and after waiting for the time interval value, continuing to acquire the next replication delay value.
N is any positive number, and may be determined according to actual conditions, which is not specifically limited in the embodiments of the present application. The minimum time interval value may be determined according to actual conditions, and this is not specifically limited in this embodiment of the present application.
S304: and acquiring a replication delay value between the current main node and the auxiliary node according to the delay acquisition time interval.
S305: and comparing the copy delay value with a preset timeout delay threshold value.
S306: and if the replication delay value is not greater than the preset timeout delay threshold value, acquiring the time of the last execution of the data manipulation language of the service request.
S307: and determining the routing mode of the service request according to the current time and the time of executing the data manipulation language last time.
The implementation manners of steps S304-S307 are similar to the implementation manners of steps S202-S205, and are not described herein again in this embodiment of the present application.
Here, in the embodiment of the present application, the delayed obtaining time interval may be automatically determined by the current replication delay value, the current time interval, and the last replication delay value, and the frequency of obtaining the delay value is automatically adjusted, so as to further improve the efficiency of database access.
Fig. 4 is a schematic structural diagram of a database access device according to an embodiment of the present application, and as shown in fig. 4, the database access device according to the embodiment of the present application includes: a first obtaining module 401, a second obtaining module 402, a first comparing module 403, a third obtaining module 404 and a first determining module 405. The database access means here may be the above-mentioned processor itself, or a chip or an integrated circuit that implements the functions of the processor. It should be noted here that the division of the first obtaining module 401, the second obtaining module 402, the first comparing module 403, the third obtaining module 404, and the first determining module 405 is only a division of one logic function, and the two may be integrated or independent physically.
The first acquisition module is used for acquiring a service request and judging whether the service request is an inquiry request;
the second acquisition module is used for acquiring a replication delay value between the current main node and the auxiliary node according to the delay acquisition time interval if the service request is the query request;
the first comparison module is used for comparing the copy delay value with a preset overtime delay threshold value;
a third obtaining module, configured to obtain a time for executing the data manipulation language of the service request at the latest time if the replication delay value is not greater than the preset timeout delay threshold value;
and the first determining module is used for determining the routing mode of the service request according to the current time and the time of executing the data manipulation language last time.
Optionally, before the second obtaining module obtains the replication delay value between the current primary node and the secondary node according to the delay obtaining time interval, the apparatus further includes:
the fourth acquisition module is used for acquiring the current replication delay value, the current time interval and the last replication delay value;
and the second determining module is used for determining the delay acquisition time interval according to the current copy delay value, the current time interval and the last copy delay value.
Optionally, the second determining module is specifically configured to:
comparing the current replica delay value with the last replica delay value;
if the current replication delay value is smaller than the last replication delay value, determining that the first delay acquisition time interval is one half of the current time interval;
if the current replication delay value is not smaller than the last replication delay value, determining that the first delay obtaining time interval is the sum of the current time interval and a preset additional value;
comparing the first delayed acquisition time interval with a preset minimum time interval;
if the first delayed acquisition time interval is smaller than a preset minimum time interval, determining the delayed acquisition time interval as the preset minimum time interval;
and if the first delayed acquisition time interval is not smaller than the preset minimum time interval, determining the delayed acquisition time interval as the first delayed acquisition time interval.
Optionally, the first determining module is specifically configured to:
determining a difference between a current time and a time at which the data manipulation language was last executed;
if the difference value is smaller than a preset routing node reading time threshold value, routing the service request to the current main node;
and if the difference value is not less than the preset routing reading node time threshold value, routing the service request to the auxiliary node.
Optionally, the first determining module is further specifically configured to:
if the difference value is smaller than a preset routing node reading time threshold value, routing the service request to the current main node;
and recording the table name and the operation completion time of the business statement.
Optionally, after the first comparing module compares the copy delay value with a preset timeout delay threshold, the apparatus further includes:
and the processing module is used for routing the service request to the current main node if the replication delay value is greater than the preset timeout delay threshold value.
Optionally, the third processing module is specifically configured to:
analyzing the service request and extracting the table related to the service request, acquiring the current time and recording the time of table operation completion;
recording the time when the table operation is completed in a global record variable;
and acquiring the time of the last execution of the data manipulation language of the service request according to the global record variable.
Fig. 5 is a schematic structural diagram of a database access device according to an embodiment of the present application, where the database access device may be the processor 102 in fig. 1. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not limiting to the implementations of the present application described and/or claimed herein.
As shown in fig. 5, the database access device includes: a processor 501 and a memory 502, the various components being interconnected using different buses, and may be mounted on a common motherboard or in other manners as desired. The processor 501 may process instructions executed within the database access device, including instructions for graphical information stored in or on a memory for display on an external input/output device (such as a display device coupled to the interface). In other embodiments, multiple processors and/or multiple buses may be used, along with multiple memories and multiple memories, as desired. In fig. 5, one processor 501 is taken as an example.
The memory 502, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules corresponding to the method of the database access apparatus in the embodiments of the present application (for example, as shown in fig. 4, the first obtaining module 401, the second obtaining module 402, the first comparing module 403, the third obtaining module 404, and the first determining module 405). The processor 501 executes various functional applications and database access methods, i.e., methods of implementing the database access device in the above-described method embodiments, by executing non-transitory software programs, instructions, and modules stored in the memory 502.
The database access device may further include: an input device 503 and an output device 504. The processor 501, the memory 502, the input device 503 and the output device 504 may be connected by a bus or other means, and fig. 5 illustrates the connection by a bus as an example.
The input device 503 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the database access apparatus, such as a touch screen, a keypad, a mouse, or a plurality of mouse buttons, a trackball, a joystick, and the like. The output device 504 may be an output device such as a display device of the database access device. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device can be a touch screen.
The database access device of the embodiment of the present application may be configured to execute the technical solutions in the method embodiments of the present application, and the implementation principle and the technical effect are similar, which are not described herein again.
The embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, the computer-readable storage medium is configured to implement the database access method according to any one of the foregoing items.
An embodiment of the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the computer program is configured to implement the database access method of any one of the foregoing methods.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a unit is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims (11)

1. A database access method, comprising:
acquiring a service request, and judging whether the service request is an inquiry request;
if the service request is a query request, acquiring a replication delay value between the current main node and the auxiliary node according to a delay acquisition time interval;
comparing the copy delay value with a preset timeout delay threshold value;
if the replication delay value is not greater than the preset timeout delay threshold value, acquiring the time of executing the data manipulation language of the service request at the latest time;
and determining the routing mode of the service request according to the current time and the time of executing the data manipulation language last time.
2. The method of claim 1, wherein before the obtaining the replica delay value between the current primary node and the secondary node according to the delay obtaining time interval, further comprising:
acquiring a current replication delay value, a current time interval and a last replication delay value;
and determining a delay acquisition time interval according to the current replication delay value, the current time interval and the last replication delay value.
3. The method of claim 2, wherein determining a delay acquisition time interval based on the current replica delay value, the current time interval, and the last replica delay value comprises:
comparing the current replica delay value and the last replica delay value;
if the current replication delay value is smaller than the last replication delay value, determining that a first delay obtaining time interval is one half of the current time interval;
if the current replication delay value is not smaller than the last replication delay value, determining a first delay obtaining time interval as the sum of the current time interval and a preset additional value;
comparing the first delayed acquisition time interval to a preset minimum time interval;
if the first delayed acquisition time interval is smaller than the preset minimum time interval, determining the delayed acquisition time interval as the preset minimum time interval;
and if the first delayed acquisition time interval is not smaller than the preset minimum time interval, determining the delayed acquisition time interval as the first delayed acquisition time interval.
4. The method according to any one of claims 1 to 3, wherein the determining the routing manner of the service request according to the current time and the time of the last execution of the data manipulation language comprises:
determining a difference between the current time and the time at which the data manipulation language was last executed;
if the difference value is smaller than a preset routing node reading time threshold value, routing the service request to the current main node;
and if the difference value is not smaller than a preset routing reading node time threshold value, routing the service request to the auxiliary node.
5. The method of claim 4, wherein the routing the service request to the current master node if the difference is less than a preset routing read node time threshold comprises:
if the difference value is smaller than a preset routing node reading time threshold value, routing the service request to the current main node;
and recording the table name and the operation completion time of the service statement.
6. The method according to any of claims 1 to 3, wherein after comparing the replica delay value with a preset timeout delay threshold, further comprising:
and if the replication delay value is greater than the preset timeout delay threshold value, routing the service request to the current main node.
7. The method according to any one of claims 1 to 3, wherein the obtaining of the time of the last execution of the data manipulation language of the service request comprises:
analyzing the service request and extracting the table related to the service request, acquiring the current time and recording the time of table operation completion;
recording the time when the table operation is completed in a global record variable;
and acquiring the time of the last execution of the data manipulation language of the service request according to the global record variable.
8. A database access apparatus, comprising:
the first acquisition module is used for acquiring a service request and judging whether the service request is an inquiry request or not;
a second obtaining module, configured to obtain a replication delay value between the current primary node and the secondary node according to a delay obtaining time interval if the service request is an inquiry request;
the first comparison module is used for comparing the replication delay value with a preset timeout delay threshold value;
a third obtaining module, configured to obtain a time for executing a data manipulation language of the service request at a latest time if the replication delay value is not greater than the preset timeout delay threshold;
and the first determining module is used for determining the routing mode of the service request according to the current time and the time of executing the data manipulation language last time.
9. A database access device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the database access method of any one of claims 1 to 7.
10. A computer-readable storage medium having stored therein computer-executable instructions for implementing the database access method of any one of claims 1 to 7 when executed by a processor.
11. A computer program product comprising a computer program, characterized in that the computer program realizes the method of any of claims 1 to 7 when executed by a processor.
CN202210486427.1A 2022-05-06 2022-05-06 Database access method, device, equipment and storage medium Pending CN114880336A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210486427.1A CN114880336A (en) 2022-05-06 2022-05-06 Database access method, device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210486427.1A CN114880336A (en) 2022-05-06 2022-05-06 Database access method, device, equipment and storage medium

Publications (1)

Publication Number Publication Date
CN114880336A true CN114880336A (en) 2022-08-09

Family

ID=82673066

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210486427.1A Pending CN114880336A (en) 2022-05-06 2022-05-06 Database access method, device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN114880336A (en)

Similar Documents

Publication Publication Date Title
US20230144450A1 (en) Multi-partitioning data for combination operations
US11580107B2 (en) Bucket data distribution for exporting data to worker nodes
US11151137B2 (en) Multi-partition operation in combination operations
US10740352B2 (en) Cooperative naming for configuration items in a distributed configuration management database environment
US8756196B2 (en) Propagating tables while preserving cyclic foreign key relationships
CN1959676B (en) Apparatus and method for materialized query table journaling in a computer database system
CN110162512B (en) Log retrieval method, device and storage medium
CN110019469B (en) Distributed database data processing method and device, storage medium and electronic device
CN109885642B (en) Hierarchical storage method and device for full-text retrieval
WO2022095366A1 (en) Redis-based data reading method and apparatus, device, and readable storage medium
CN114356921A (en) Data processing method, device, server and storage medium
CN111949402A (en) Database request processing method and device, computer equipment and storage medium
CN111651424B (en) Data processing method, device, data node and storage medium
CN111414356A (en) Data storage method and device, non-relational database system and storage medium
CN106815318B (en) Clustering method and system for time sequence database
CN116701413A (en) Main data processing method and device
CN116186082A (en) Data summarizing method based on distribution, first server and electronic equipment
CN114880336A (en) Database access method, device, equipment and storage medium
EP3082050A1 (en) Mass data fusion storage method and system
CN108664503A (en) A kind of data archiving method and device
CN111078736A (en) Data aggregation processing method and device, terminal and storage medium
US20230136417A1 (en) Database system, and transaction management method and apparatus
CN111797062B (en) Data processing method, device and distributed database system
US20240104077A1 (en) Data storage method, query method, electronic device and readable medium
CN110633274B (en) Alarm management method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination