CN117493298A - Database management method and device, electronic equipment and storage medium - Google Patents

Database management method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN117493298A
CN117493298A CN202310654174.9A CN202310654174A CN117493298A CN 117493298 A CN117493298 A CN 117493298A CN 202310654174 A CN202310654174 A CN 202310654174A CN 117493298 A CN117493298 A CN 117493298A
Authority
CN
China
Prior art keywords
slave
database
delay
master
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310654174.9A
Other languages
Chinese (zh)
Inventor
卿力
赵飞
罗仕杰
吴海英
蒋宁
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mashang Xiaofei Finance Co Ltd
Original Assignee
Mashang Xiaofei Finance Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mashang Xiaofei Finance Co Ltd filed Critical Mashang Xiaofei Finance Co Ltd
Priority to CN202310654174.9A priority Critical patent/CN117493298A/en
Publication of CN117493298A publication Critical patent/CN117493298A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21Design, administration or maintenance of databases
    • G06F16/211Schema design and management
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28Databases characterised by their database models, e.g. relational or object models
    • G06F16/284Relational databases

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Computing Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

One or more embodiments of the present disclosure provide a database management method and apparatus, an electronic device, and a storage medium, where the method includes: obtaining a master-slave delay between a master database and a slave database, wherein the master-slave delay characterizes a time difference between a write time of the same data in the master database and a write time in the slave database; and determining the data source of each data query scene in the master database and the slave database according to the master-slave delay and the delay threshold of each data query scene in the plurality of data query scenes, wherein the data source of each data query scene is the master database or the slave database, and the delay threshold is used for representing the tolerance degree of the data query scene to the data delay. The method can ensure that the returned data in each data query scene is within the tolerance degree, namely, each service can ensure the service quality when the database has faults such as delay and the like.

Description

Database management method and device, electronic equipment and storage medium
Technical Field
One or more embodiments of the present disclosure relate to the field of database technologies, and in particular, to a method and apparatus for managing a database, an electronic device, and a storage medium.
Background
Today, the development of the internet and informatization is rapid, and the generation of data is explosively increasing, so that the requirements for databases and management thereof are increasing. The database cluster is composed of a plurality of databases, and different databases can be utilized to provide different services, such as read-write separation technology and the like, so that the pressure of each database is reduced. However, in the related art, the mapping relationship between the database and the service in the database cluster is often fixed, and when a failure such as a delay occurs in the database providing a certain service, the quality of the service is degraded, and even the service cannot be provided.
Disclosure of Invention
In view of this, one or more embodiments of the present disclosure provide a database management method and apparatus, an electronic device, and a storage medium.
In order to achieve the above object, one or more embodiments of the present disclosure provide the following technical solutions:
according to a first aspect of one or more embodiments of the present specification, there is provided a database management method, including:
obtaining a master-slave delay between a master database and a slave database, wherein the master-slave delay characterizes a time difference between a write time of the same data in the master database and a write time in the slave database;
And determining the data source of each data query scene in the master database and the slave database according to the master-slave delay and the delay threshold of each data query scene in the plurality of data query scenes, wherein the data source of each data query scene is the master database or the slave database, and the delay threshold is used for representing the tolerance degree of the data query scene to the data delay.
According to a second aspect of one or more embodiments of the present specification, there is provided a database management apparatus comprising:
the acquisition module is used for acquiring master-slave delay between a master database and a slave database, wherein the master-slave delay represents the time difference between the writing time of the same data in the master database and the writing time in the slave database;
the determining module is used for determining the data source of each data query scene in the master database and the slave database according to the master-slave delay and the delay threshold value of each data query scene in the plurality of data query scenes, wherein the data source of each data query scene is the master database or the slave database, and the delay threshold value is used for representing the tolerance degree of the data query scene to the data delay.
According to a third aspect of one or more embodiments of the present specification, there is provided an electronic device comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor implements the method of the first aspect by executing the executable instructions.
According to a fourth aspect of one or more embodiments of the present description, there is provided a computer readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the steps of the method as described in the first aspect.
The technical scheme provided by the embodiment of the specification can comprise the following beneficial effects:
in the database management method provided by the embodiment of the specification, master-slave delay between a master database and a slave database is obtained, and then a data source of each data query scene is determined in the master database and the slave database according to the master-slave delay and a delay threshold of each data query scene in a plurality of data query scenes. The method can independently judge the data source of each data inquiry scene when the master-slave delay changes, namely, the current data source is kept or the data source is switched to other data sources, so that the data returned under each data inquiry scene is within the tolerance degree, namely, each service can ensure the service quality when the database has faults such as delay and the like.
Drawings
Fig. 1 is a schematic diagram of an application environment of a database management method according to an exemplary embodiment.
Fig. 2 is a schematic diagram of an application scenario of a database management method according to an exemplary embodiment.
Fig. 3 is a flowchart of a database management method according to an exemplary embodiment.
Fig. 4 is a flowchart of a database management method according to an exemplary embodiment.
Fig. 5 is a schematic diagram of an apparatus according to an exemplary embodiment.
Fig. 6 is a block diagram of a database management apparatus according to an exemplary embodiment.
Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of the present specification. Rather, they are merely examples of apparatus and methods consistent with aspects of one or more embodiments of the present description as detailed in the accompanying claims.
It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than described in this specification. Furthermore, individual steps described in this specification, in other embodiments, may be described as being split into multiple steps; while various steps described in this specification may be combined into a single step in other embodiments.
As mentioned above, the mapping relationship between the databases in the database cluster and the service is usually fixed, for example, by means of a configuration file, and when a failure such as a delay occurs in the database providing a certain service, the quality of the service is degraded, and even the service cannot be provided. Of course, in the related art, when the situation occurs, the data source of the service can be manually switched, but the timeliness, the accuracy and the automation degree are poor, and the technical problem cannot be solved.
In view of the above problems, the inventor has found through long-term researches that the database management method, the device, the electronic equipment, the storage medium and the program product provided in the embodiments of the present application divide the data query service responsible for the database cluster into a plurality of data query scenarios according to the tolerance degree of the data delay, that is, each data query scenario has different tolerance degrees to the data delay, for example, the data delay that can be tolerated by the data query scenario a is a, the data delay that can be tolerated by the data query scenario B is B, and the like, and determines the data delay that can be tolerated by each scenario as the delay threshold of the corresponding scenario; based on the delay threshold value of each data query scene, the method can acquire master-slave delay between a master database and a slave database in a database cluster according to a certain frequency, and compare the master-slave delay with the delay threshold value of each data query scene to judge whether the master-slave delay is within the tolerance degree of each data query scene or not, and further dynamically adjust the data source of each data query scene, thereby changing the fixed mode of service and database in the related technology, solving the technical problems of service quality reduction or incapability of providing service caused by the fixed mode, and ensuring that the returned data of each data query scene is within the tolerance degree, namely ensuring the service quality when the database has faults such as delay and the like.
Referring to fig. 1, an application environment of a database management method provided in the present application is shown as an example, and as an implementation manner, the database management method provided in the embodiment of the present application may be applied to a decision query application of a cluster server in fig. 1, where the decision query application determines a data source for each data query scenario of a service system. In fig. 1, a cluster server is connected to clients through a network, which is a medium used to provide a communication link between the cluster server and the clients, and the network may include various connection types, such as a wired communication link, a wireless communication link, and the like, which is not limited by the embodiments of the present application. In fig. 1, the cluster Server may include a database cluster, a decision query application, a plurality of application instances, and the like, where the database cluster may be an MGR cluster (a distributed cluster formed by a plurality of MySQL Server nodes together), and includes a master database and a plurality of slave databases as shown in fig. 1, where the master database may provide a writing service externally, that is, after the application instance receives a data writing request sent by a client, a writing action is performed on the master database, and data synchronization is performed between the master database and each slave database in real time, so that each data writing of the master database is synchronized to each slave database by a log replay manner, which inevitably causes some delay between data in the slave databases and the master database; the master database and the slave database can provide reading service to the outside, namely, after an application instance receives a data reading request sent by a client, the application instance judges a data source of the data reading request and executes reading action aiming at the data source; each slave database may be configured with a monitoring agent, and each monitoring agent may be communicatively connected (e.g., interfaced) with the decision query application, and each monitoring agent may be configured to monitor master-slave delays, number of delayed transactions, etc. of the slave database at which it is located and report to the decision query application; the decision query application may be communicatively coupled to each application instance to notify the application instance when the data source is determined or switched. It is to be appreciated that the decision query application and the plurality of application instances can run on the same electronic device or on different electronic devices separately.
The database management method provided by the application can be applied to business scenes such as electric marketing and the like shown in fig. 2, in the business scenes, a user inputs a data query request through a client, the client sends the data query request to a cluster server through a network, the cluster server determines a data source in a database cluster and executes query action aiming at the data source, and finally, a query result is returned to a value client for the user to check.
Referring to fig. 3, a flowchart of a database management method provided in the present application is shown, and includes steps S301 to S302.
In step S301, a master-slave delay between a master database and a slave database is obtained, wherein the master-slave delay characterizes a time difference between a write time of the same data in the master database and a write time in the slave database.
Wherein the data synchronization between the master database and the slave database requires time, so there is a delay between the two, which is called master-slave delay; the step can accurately acquire master-slave delay. It will be appreciated that in the case where the database cluster includes a plurality of slave databases, this step may obtain a master-slave delay between the master database and each of the plurality of databases; various ways of determining master-slave delays described below may be performed in parallel between the master database and each slave database to determine master-slave delays between the master database and each slave database, respectively. Further, after determining the master-slave delay between the master database and each slave database, an average value of all master-slave delays may be further calculated, and a difference between each master-slave delay and the average value may be calculated, and if the difference between a certain master-slave delay and the average value is continuously higher than an alarm threshold for a certain period of time (e.g., 10 s), alarm information may be generated to prompt the user to check and maintain the slave database to which the master-slave delay belongs.
This step may be performed when the cluster server has a data query service; or at a frequency such that the master-slave delay is obtained once every predetermined time period (e.g., 1 s).
Illustratively, this step may obtain the master-slave delay between the master database and the slave database as follows: firstly, writing delay data into the main database and marking writing time for the delay data, wherein the delay data is data for calculating master-slave delay; and next, acquiring the master-slave delay determined by a monitoring agent in the slave database, wherein the monitoring agent is used for reading the delay data in the slave database and determining the master-slave delay according to the current time and the writing time marked on the delay data. For example, a delay table for recording delay data may be provided in the main database, the table may include at least a field of writing time, and the decision query application executing the method may write the current time into the writing time field of the delay table of the main database, thereby completing the writing of delay data and the marking of writing time; the slave database is internally provided with the same delay table as the master database, delay data written in the delay table of the master database and the writing time of the mark thereof are synchronized to the slave database, the monitoring agent in the slave database reads the delay data, and the time difference between the writing time of the mark on the delay data and the current time is determined as master-slave delay; the monitoring agent may report the determined master-slave delay to the decision query application, or the decision query application actively obtains the master-slave delay determined by the monitoring agent.
Optionally, when the step is executed at intervals of a preset time length, writing a piece of delay data into the main database at intervals of the preset time length, and marking writing time for each piece of delay data, wherein the delay data is used for calculating master-slave delay; and then acquiring the master-slave delay which is newly determined by the monitoring agent in the slave database at each interval of the preset duration, wherein the monitoring agent is used for reading the latest delay data in the slave database at each interval of the preset duration, and determining the master-slave delay according to the current time and the writing time marked on the latest delay data. The details of the operations of writing delay data, marking writing time, reading delay data, determining master-slave delay, etc. are described in more detail above, and the detailed description thereof is omitted here.
In the above example, by writing data into the master database and marking the writing time, then reading the data from the slave database and calculating the time difference as the master-slave delay, the master-slave delay between the master database and the slave database can be accurately determined in real time, thereby facilitating decision query application and decision data source between the master database and the slave database.
In step S302, determining a data source of each data query scene in the master database and the slave database according to the master-slave delay and a delay threshold of each data query scene in the plurality of data query scenes, where the data source of each data query scene is the master database or the slave database, and the delay threshold is used to represent tolerance (or acceptance) of the data query scene to the data delay.
Because the step S301 accurately calculates the master-slave delay, the step can evaluate the validity of the quasi-real-time data by combining the master-slave delay with the threshold value of each scene of the service system, and automatically make a decision about the data source of each scene.
It should be noted that, in the case where the database cluster includes a plurality of slave databases, the step may determine, in the slave databases corresponding to the master database and the minimum master-slave delay, a data source of each data query scene according to a minimum master-slave delay and a delay threshold of each data query scene in the plurality of data query scenes, where the minimum master-slave delay includes a minimum master-slave delay among the plurality of master-slave delays between the plurality of slave databases and the master database. That is, the decision query application executing the method may compare after obtaining the master-slave delay between the master database and each slave database, and pick the slave database with the smallest master-slave delay, and then select the final data source from the master database and the slave database with the smallest master-slave delay in this step.
It can be appreciated that the data query scenarios are different scenarios in the service, each of which is tolerant to data delay differently. For example, the electric pin business has data query scenes such as list dialing, list distribution, list recovery and the like, wherein the tolerance degree of the list dialing, the list distribution and the list recovery to the data delay is sequentially reduced, for example, the delay threshold of the list dialing is 30s when the list dialing can accept the delay of up to 30s, the delay threshold of the list distribution is 20s when the list distribution can accept the delay of up to 20s, and the delay threshold of the list recovery is 10s when the list recovery can accept the delay of up to 10s. In one example, a master-slave query persistence maintenance data table for each scenario may be established to store the delay threshold, e.g., in key: value, map mode stores a delay threshold table:
{
"list dialing": { "trigger_delay":30},
"List assignment" { "trigger_delay";20},
"List reclamation" { "trigger_delay":10},
}
the data without delay is real-time data and the data with a certain delay may be referred to as near real-time data. If the delay threshold of a certain data query scene is 0, the data query scene requires the data to be real-time data, so that the data source can only be a main database, and the method is not needed to be used for determining the data source; in other words, the method mainly aims at data query scenes requiring that data are quasi-real-time data, the data sources of the data query scenes are initialized to the slave database, and the slave database is switched to the master database to be the data sources when the delay of the slave database relative to the master database is large, so that the delay of the queried data is prevented from exceeding a delay threshold value.
Illustratively, the step may determine that the data source of the data query scenario is the primary database in response to the master-slave delay being greater than a delay threshold of the data query scenario; and determining that the data source of the data query scenario is a slave database in response to the master-slave delay not being greater than a delay threshold of the data query scenario.
It will be appreciated that, to avoid that such transient variations affect the accuracy of the data source, when it is determined in this step that the data source of a certain data query scene changes, for example, when the data source is switched from the master database to the slave database, the data source of the data query scene is determined by continuing to use this step for a certain period of time (for example, 3 s), and if the data source of the data query scene is continuously maintained as the slave database for this certain period of time, the data source of the data query scene is determined as the slave database.
The decision query application executing the method can maintain a scene queue to be restored based on the data source of each data query scene determined in the step; as the name suggests, a scene in the to-be-restored scene queue is a scene that waits to restore the data source to the default (i.e., initialized) data source. Specifically, the decision query application may add a data query scenario with a data source being the master database to a to-be-restored scenario queue, and remove a data query scenario with a data source being the slave database in the to-be-restored scenario queue. Thereby facilitating data source tagging of various data query scenarios.
In the database management method provided by the embodiment of the specification, master-slave delay between a master database and a slave database is obtained, and then a data source of each data query scene is determined in the master database and the slave database according to the master-slave delay and a delay threshold of each data query scene in a plurality of data query scenes. The method can independently judge the data source of each data inquiry scene when the master-slave delay changes, namely, the current data source is kept or the data source is switched to other data sources, so that the data returned under each data inquiry scene is within the tolerance degree, namely, each service can ensure the service quality when the database has faults such as delay and the like, and the stability of a business system is improved; moreover, the method has higher automation degree and does not need manual intervention.
In some embodiments of the present disclosure, the monitoring agent in the slave database may also monitor the number of delayed transactions in the slave database, so the decision query application executing the method may also obtain the number of delayed transactions in the slave database, and in determining the data source of each data query scenario, determine the data source of each data query scenario in combination with the number of delayed transactions and the master-slave delay dimension. The number of the delayed transactions can be actively reported by the monitoring agent or actively acquired by the decision query application.
It will be appreciated that where the database cluster includes a plurality of slave databases, a decision query application executing the method may obtain the number of delayed transactions within each slave database and determine the data source for each data query scenario in combination with the total number of delayed transactions within all slave databases and the two dimensions of the master and slave data.
By way of example, the data source for each data poll scenario may be determined in conjunction with the number of delayed transactions and the master-slave delay dimension in the following manner: determining that a data source of the data query scene is a master database in response to the number of delayed transactions being greater than a number threshold and the master-slave delay being greater than a delay threshold of the data query scene; and determining that the data source of the data query scene is a slave database in response to the number of delayed transactions being less than a number threshold or the master-slave delay not being greater than a delay threshold of the data query scene.
Further exemplary, the data sources for each data query scenario may be determined in conjunction with the number of delayed transactions and the master-slave delay dimension in the following manner: determining that the data source of the data query scenario is a slave database in response to the number of delayed transactions being not greater than a number threshold; responsive to the number of delayed transactions being greater than a number threshold, comparing a master-slave delay to a delay threshold of the data query scenario; and if the master-slave delay is not greater than the delay threshold of the data query scene, determining that the data source of the data query scene is a slave database.
In this embodiment, the number of delay transactions and the master-slave delay are combined to determine the data source, so that the delay condition between the master database and the slave database can be evaluated more accurately, thereby ensuring that the data searched by each data searching scene is within the tolerance degree, and reducing the access pressure of the master database as much as possible.
In some embodiments of the present disclosure, after determining the data source of each data query scenario, the decision query application executing the method may further determine whether the data source of each data query scenario changes, if not, no further operation is required, and if so, the application instance may be notified, so that the application instance switches the data sources when executing the data query. Specifically, in response to switching of a data source of any data query scene from the master database to the slave database, sending a first data source switching notification to at least one application instance, wherein the first data source switching notification is used for instructing the at least one application instance to execute a query action under the data query scene for the data source from the slave database; and responding to the data source of any data query scene, switching from the slave database to the master database, and sending a second data source switching notification to at least one application instance, wherein the second data source switching notification is used for indicating the at least one application instance to execute the query action under the data query scene for the data source by the master database.
The decision query application may send a data source switching notification to at least one application instance in a redis event subscription manner and/or an eureka registration ip manner. For example, the decision query application may preferably notify each application instance in a redis event subscription manner, and if a redis occurs in a special exception or other scenario, the notification manner may be downgraded into a notification manner that each application instance registers ip for the eureka.
The first data source switching notification can also be used for indicating master-slave delay of synchronous data return when the application instance returns data to the client, so that the client can display the query result and delay of the query result, and a user (for example, an electric sales agent in an electric sales service scene) can conveniently judge the data value of the query result according to the delay, and timeliness of the data is improved. Specifically, in response to switching of a data source of any data query scene from the master database to the slave database, a first data source switching notification is sent to at least one application instance, and the first data source switching notification is used for instructing the at least one application instance to return data for the query action to a client, and simultaneously, the master-slave delay is returned to the client.
In this embodiment, the decision query application synchronizes the latest data source to the application instance in time, so that the application instance can adjust the data source in time, and ensure that the data queried by the application instance meets the delay requirement of the corresponding scene, thereby further avoiding the influence of the database delay on the service. And the decision query application is timely synchronized to the application instance when the data source is switched to the slave database, so that the application instance can adjust the response action of the query request while adjusting the data source, namely, increase the master-slave delay of the data in the returned content, so that a user (for example, an electric pin agent in an electric pin service scene) can judge the data value of the query result according to the delay, the marketing effect of the agent based on the data is ensured in the electric pin service scene, and the data value is exerted to the maximum.
Referring to fig. 4, a flowchart of a database management method obtained by combining the above embodiments is shown, and a detailed description of some steps in fig. 4 is provided.
Whether a recovery scene exists or not refers to: whether a data query scene with a delay threshold larger than the master-slave delay exists in the scene queue to be restored or not, and if so, the scene is a restoration scene.
The step of closing the degradation switch by each application instance means that: each scene data source defaulted by each application instance is a slave database, if the master-slave delay is greater than the delay threshold of a certain scene, the application instance opens a degradation switch for the scene, and the degradation data source is a master database; the demotion switch is turned off when the recoverable data source is a slave database.
Fig. 5 is a schematic block diagram of an apparatus according to an exemplary embodiment. Referring to fig. 5, at the hardware level, the device includes a processor 502, an internal bus 504, a network interface 506, a memory 508, and a non-volatile storage 510, although other tasks may be performed by the device. One or more embodiments of the present description may be implemented in a software-based manner, such as by the processor 502 reading a corresponding computer program from the non-volatile storage 510 into the memory 508 and then running. Of course, in addition to software implementation, one or more embodiments of the present disclosure do not exclude other implementation manners, such as a logic device or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but may also be hardware or a logic device.
Referring to fig. 6, the database management apparatus may be applied to the device shown in fig. 5 to implement the technical solution of the present specification. The device comprises:
an obtaining module 601, configured to obtain a master-slave delay between a master database and a slave database, where the master-slave delay characterizes a time difference between a write time of the same data in the master database and a write time in the slave database;
the determining module 602 is configured to determine, according to the master-slave delay and a delay threshold of each data query scenario among the plurality of data query scenarios, a data source of each data query scenario in the master database and the slave database, where the data source of each data query scenario is the master database or the slave database, and the delay threshold is used to characterize tolerance of the data query scenario to the data delay.
In one embodiment of the present specification, the obtaining module is specifically configured to:
writing delay data into the main database and marking writing time for the delay data, wherein the delay data is used for calculating master-slave delay;
and acquiring the master-slave delay determined by the monitoring agent in the slave database, wherein the monitoring agent is used for reading the delay data in the slave database and determining the master-slave delay according to the current time and the writing time marked on the delay data.
In one embodiment of the present specification, the obtaining module is specifically configured to:
writing a piece of delay data into the main database at each interval of the preset time length, and marking writing time for each piece of delay data, wherein the delay data is used for calculating master-slave delay;
and acquiring the master-slave delay which is newly determined by the monitoring agent in the slave database at each interval of the preset duration, wherein the monitoring agent is used for reading the latest delay data in the slave database at each interval of the preset duration, and determining the master-slave delay according to the current time and the writing time marked on the latest delay data.
In one embodiment of the present specification, the determining module is specifically configured to:
determining a data source of the data query scene as a master database in response to the master-slave delay being greater than a delay threshold of the data query scene;
and determining that the data source of the data query scene is a slave database in response to the master-slave delay not being greater than a delay threshold of the data query scene.
In one embodiment in the present specification, the method further includes a transaction module for:
Acquiring the number of delayed transactions in the slave database;
the determining module is specifically configured to:
determining the data source of each data query scene in the master database and the slave database according to the number of the delay transactions, the master-slave delay and the delay threshold value of each data query scene in a plurality of data query scenes.
In one embodiment of the present specification, the determining module is configured to determine, when determining the data source of each data query scenario in the master database and the slave database, according to the number of the delayed transactions, the master-slave delay, and a delay threshold of each data query scenario in the plurality of data query scenarios, specifically configured to:
determining that a data source of the data query scene is a master database in response to the number of delayed transactions being greater than a number threshold and the master-slave delay being greater than a delay threshold of the data query scene;
and determining that the data source of the data query scene is a slave database in response to the number of delayed transactions being less than a number threshold or the master-slave delay not being greater than a delay threshold of the data query scene.
In one embodiment of the present specification, the obtaining module is specifically configured to:
Acquiring master-slave delay between the master database and each slave database in a plurality of slave databases;
the determining module is specifically configured to:
determining a data source of each data query scene in a slave database corresponding to the master database and the minimum master-slave delay according to the minimum master-slave delay and a delay threshold of each data query scene in a plurality of data query scenes, wherein the minimum master-slave delay comprises the minimum master-slave delay in a plurality of master-slave delays between the plurality of slave databases and the master database.
In one embodiment of the present specification, the apparatus further includes a queue module configured to:
after the data source of each data query scene is determined, adding the data query scene with the data source being the main database into a scene queue to be restored, and removing the data query scene with the data source being the slave database in the scene queue to be restored.
In one embodiment of the present specification, the apparatus further includes a notification module configured to:
after determining the data source of each data query scene, responding to the data source of any data query scene to be switched from the master database to the slave database, and sending a first data source switching notification to at least one application instance, wherein the first data source switching notification is used for instructing the at least one application instance to return the data for the query action to the client and simultaneously returning the master-slave delay to the client.
The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular telephone, camera phone, smart phone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or a combination of any of these devices.
In a typical configuration, a computer includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, read only compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic disk storage, quantum memory, graphene-based storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by the computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
The foregoing describes specific embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
The terminology used in the one or more embodiments of the specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of the specification. As used in this specification, one or more embodiments and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.
User information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, presented data, etc.) referred to herein are both user-authorized or fully authorized information and data by parties, and the collection, use and processing of relevant data requires compliance with relevant laws and regulations and standards of the relevant country and region, and is provided with corresponding operation portals for user selection of authorization or denial.
It should be understood that although the terms first, second, third, etc. may be used in one or more embodiments of the present description to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of one or more embodiments of the present description. The word "if" as used herein may be interpreted as "at … …" or "at … …" or "responsive to a determination", depending on the context.
The foregoing description of the preferred embodiment(s) is (are) merely intended to illustrate the embodiment(s) of the present invention, and it is not intended to limit the embodiment(s) of the present invention to the particular embodiment(s) described.

Claims (12)

1. A method of database management, comprising:
obtaining a master-slave delay between a master database and a slave database, wherein the master-slave delay characterizes a time difference between a write time of the same data in the master database and a write time in the slave database;
and determining the data source of each data query scene in the master database and the slave database according to the master-slave delay and the delay threshold of each data query scene in the plurality of data query scenes, wherein the data source of each data query scene is the master database or the slave database, and the delay threshold is used for representing the tolerance degree of the data query scene to the data delay.
2. The method of claim 1, wherein the obtaining a master-slave delay between the master database and the slave database comprises:
Writing delay data into the main database and marking writing time for the delay data, wherein the delay data is used for calculating master-slave delay;
and acquiring the master-slave delay determined by the monitoring agent in the slave database, wherein the monitoring agent is used for reading the delay data in the slave database and determining the master-slave delay according to the current time and the writing time marked on the delay data.
3. The method of claim 1, wherein the obtaining a master-slave delay between the master database and the slave database comprises:
writing a piece of delay data into the main database at each preset interval time length, and marking writing time for each piece of delay data, wherein the delay data is used for calculating master-slave delay;
and acquiring the master-slave delay which is newly determined by the monitoring agent in the slave database at each interval of the preset duration, wherein the monitoring agent is used for reading the latest delay data in the slave database at each interval of the preset duration, and determining the master-slave delay according to the current time and the writing time marked on the latest delay data.
4. The method of claim 1, wherein determining the data source of the data query scenario in the master database and the slave database based on the master-slave delay and the delay threshold of the data query scenario comprises:
determining a data source of the data query scene as a master database in response to the master-slave delay being greater than a delay threshold of the data query scene;
and determining that the data source of the data query scene is a slave database in response to the master-slave delay not being greater than a delay threshold of the data query scene.
5. The database management method according to claim 1, wherein the method further comprises:
acquiring the number of delayed transactions in the slave database;
the determining, in the master database and the slave database, a data source of each data query scenario according to the master-slave delay and a delay threshold of each data query scenario in the plurality of data query scenarios, including:
determining the data source of each data query scene in the master database and the slave database according to the number of the delay transactions, the master-slave delay and the delay threshold value of each data query scene in a plurality of data query scenes.
6. The method of claim 5, wherein determining the data source of the data query scenario in the master database and the slave database based on the number of delayed transactions, the master-slave delay, and a delay threshold of the data query scenario comprises:
determining that a data source of the data query scene is a master database in response to the number of delayed transactions being greater than a number threshold and the master-slave delay being greater than a delay threshold of the data query scene;
and determining that the data source of the data query scene is a slave database in response to the number of delayed transactions being less than a number threshold or the master-slave delay not being greater than a delay threshold of the data query scene.
7. The method of claim 1, wherein the obtaining a master-slave delay between the master database and the slave database comprises:
acquiring master-slave delay between the master database and each slave database in a plurality of slave databases;
the determining, in the master database and the slave database, a data source of each data query scenario according to the master-slave delay and a delay threshold of each data query scenario in the plurality of data query scenarios, including:
Determining a data source of each data query scene in a slave database corresponding to the master database and the minimum master-slave delay according to the minimum master-slave delay and a delay threshold of each data query scene in a plurality of data query scenes, wherein the minimum master-slave delay comprises the minimum master-slave delay in a plurality of master-slave delays between the plurality of slave databases and the master database.
8. The database management method according to claim 1, wherein after said determining the data source of each data query scenario, the method further comprises:
and adding the data query scene with the data source being the master database into a scene queue to be restored, and removing the data query scene with the data source being the slave database in the scene queue to be restored.
9. The database management method according to claim 1, wherein after said determining the data source of each data query scenario, the method further comprises:
and responding to the data source of any data query scene, switching from the master database to the slave database, and sending a first data source switching notification to at least one application instance, wherein the first data source switching notification is used for instructing the at least one application instance to return the data for the query action to a client and simultaneously returning the master-slave delay to the client.
10. A database management apparatus, comprising:
the acquisition module is used for acquiring master-slave delay between a master database and a slave database, wherein the master-slave delay represents the time difference between the writing time of the same data in the master database and the writing time in the slave database;
the determining module is used for determining the data source of each data query scene in the master database and the slave database according to the master-slave delay and the delay threshold value of each data query scene in the plurality of data query scenes, wherein the data source of each data query scene is the master database or the slave database, and the delay threshold value is used for representing the tolerance degree of the data query scene to the data delay.
11. An electronic device, comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to implement the method of any of claims 1-9 by executing the executable instructions.
12. A computer readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the steps of the method of any of claims 1-9.
CN202310654174.9A 2023-06-02 2023-06-02 Database management method and device, electronic equipment and storage medium Pending CN117493298A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310654174.9A CN117493298A (en) 2023-06-02 2023-06-02 Database management method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310654174.9A CN117493298A (en) 2023-06-02 2023-06-02 Database management method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN117493298A true CN117493298A (en) 2024-02-02

Family

ID=89683555

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310654174.9A Pending CN117493298A (en) 2023-06-02 2023-06-02 Database management method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN117493298A (en)

Similar Documents

Publication Publication Date Title
US10439937B2 (en) Service addressing in distributed environment
CN113254466B (en) Data processing method and device, electronic equipment and storage medium
CN111177165B (en) Method, device and equipment for detecting data consistency
CN111324665B (en) Log playback method and device
CN107153644B (en) Data synchronization method and device
CN107896170B (en) Insure the monitoring method and device of application system
CN110162512B (en) Log retrieval method, device and storage medium
CN107179940B (en) Task execution method and device
CN105610917B (en) Method and system for realizing synchronous data repair in system
CN111338834B (en) Data storage method and device
CN112433921A (en) Method and apparatus for dynamic point burying
CN110928945B (en) Data processing method and device for database and data processing system
CN117493298A (en) Database management method and device, electronic equipment and storage medium
CN107154960B (en) Method and apparatus for determining service availability information for distributed storage systems
US20160085638A1 (en) Computer system and method of identifying a failure
CN114844771A (en) Monitoring method, device, storage medium and program product for micro-service system
CN115129781A (en) Data processing method and device, electronic equipment and storage medium
CN110266610B (en) Traffic identification method and device and electronic equipment
CN114003612A (en) Processing method and processing system for abnormal conditions of database
CN115695587A (en) Service data processing system, method, device and storage medium
CN106375354B (en) Data processing method and device
CN117349384B (en) Database synchronization method, system and equipment
US12001450B2 (en) Distributed table storage processing method, device and system
US20230030856A1 (en) Distributed table storage processing method, device and system
CN115766498A (en) Big data link full link tracking monitoring method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination