CN115658701A - Database flow control method, device, equipment and storage medium - Google Patents
Database flow control method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN115658701A CN115658701A CN202211680276.XA CN202211680276A CN115658701A CN 115658701 A CN115658701 A CN 115658701A CN 202211680276 A CN202211680276 A CN 202211680276A CN 115658701 A CN115658701 A CN 115658701A
- Authority
- CN
- China
- Prior art keywords
- database
- data
- flow
- flow control
- load
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention relates to the technical field of databases, and provides a database flow control method, a device, equipment and a storage medium, wherein the method comprises the following steps: acquiring historical monitoring data of a database; the historical monitoring data comprises historical flow data and historical load information of the database; carrying out iterative prediction on the load information of the database based on the historical monitoring data to obtain load prediction data of the database; and determining a flow control strategy according to the load prediction data, and controlling the flow of the database according to the flow control strategy. According to the database flow control method provided by the invention, the load trend of the database is predicted according to the historical flow data and the historical load information of the database by predicting the load information of the database, the flow control strategy is determined in advance, and the flow control is carried out on the database, so that the timeliness of the flow control is improved.
Description
Technical Field
The present invention relates to the field of database technologies, and in particular, to a method, an apparatus, a device, and a storage medium for controlling database traffic.
Background
With the development of information technology, enterprises gradually realize paperless office work by building an informatization system, and a database is produced for realizing service interconnection and intercommunication. At present, relational databases are commonly used, in general, data query requests are most frequent for databases, an upstream business system or micro-service connects the databases through query statements for data query, and when there are many requests or the amount of queried data is large, resources of a database cluster processor are consumed in a large amount, so that overall business response is slow or abnormal. The existing solution is to generally manage and control the traffic in the upstream service system or microservice, and when the upstream request suddenly increases, by reducing the processing speed of the service system or microservice itself, or by using the current limiting capability of the database cluster itself, the speed of the traffic flow flowing into the database cluster is limited, so as to prevent the downstream database cluster from being down due to the over-pressure. However, flow control is performed in an upstream service system or micro service, or the current limiting capability of the database cluster is utilized, only current limitation can be performed when the flow suddenly increases, the flow control has hysteresis, the flow control effect is poor, and great influence is easily caused on the service.
Disclosure of Invention
The invention provides a database flow control method, a database flow control device and a database flow control storage medium, which are used for solving the defects that the existing flow control mode has hysteresis on flow control, the flow control effect is poor, and great influence is easily caused on services.
The invention provides a database flow control method, which comprises the following steps:
acquiring historical monitoring data of a database; the historical monitoring data comprises historical flow data and historical load information of the database;
carrying out iterative prediction on the load information of the database based on the historical monitoring data to obtain load prediction data of the database;
and determining a flow control strategy according to the load prediction data, and controlling the flow of the database according to the flow control strategy.
According to the database flow control method provided by the invention, the iterative prediction of the load information of the database based on the historical monitoring data to obtain the load prediction data of the database comprises the following steps:
selecting monitoring data with a first preset time length before the reference time point as prediction sample data based on the historical monitoring data by taking the current time point as the reference time point;
predicting the monitoring data of the database for a second preset time after the reference time point based on the predicted sample data to obtain predicted monitoring data of the database;
performing sliding processing on the reference time point by taking the second preset time length as a sliding step length, returning and executing the step of selecting monitoring data with a first preset time length before the reference time point as prediction sample data based on the historical monitoring data, and determining load prediction data of the database with a third preset time length after the current time point according to the prediction monitoring data until the reference time point is a time point with a third preset time length after the current time point;
the first preset time is longer than the second preset time, and the third preset time is shorter than the first preset time and longer than the second preset time.
According to the database flow control method provided by the invention, the prediction sample data comprises a flow sample and a load sample; the predictive monitoring data comprises first predicted traffic and first load predictive data; the predicting the monitoring data of the database for a second preset time after the reference time point based on the prediction sample data to obtain the predicted monitoring data of the database comprises:
predicting the flow of the database for a second preset time after the reference time point based on the flow sample in the predicted sample data to obtain a first predicted flow;
and predicting load information of the database in a second preset time after the reference time point according to the first predicted flow and the load sample in the predicted sample data to obtain first load predicted data.
According to the database flow control method provided by the invention, the determining of the flow control strategy according to the load prediction data comprises the following steps:
determining a load gradient interval corresponding to the load prediction data;
determining a flow control strategy according to the load gradient interval; wherein; the load gradient interval comprises a plurality of load gradient intervals, and different load gradient intervals correspond to different flow control strategies.
According to the database flow control method provided by the invention, the flow control strategy comprises at least one of a slow query termination strategy, a slow query filtering strategy and a priority strategy; if the flow control policy includes the slow query termination policy, the performing flow control on the database according to the flow control policy includes:
acquiring current target flow data of the database; the target flow data comprises first flow data with an execution state being execution and second flow data with an execution state being to be executed;
identifying a first slow query statement with execution duration exceeding a fourth preset duration from the first flow data;
and executing a termination operation on the first slow query statement to control the flow of the database.
According to the method for controlling the flow of the database provided by the invention, if the flow control strategy comprises the slow query filtering strategy, the controlling the flow of the database according to the flow control strategy comprises the following steps:
identifying a second slow query statement from the second traffic data;
filtering the second traffic data based on the second slow query statement to flow control the database.
According to a method for controlling a database flow provided by the present invention, if the flow control policy includes the priority policy, the controlling the database flow according to the flow control policy includes:
determining the historical execution duration of each query statement in the second flow data according to the historical monitoring data;
and determining the response priority of each query statement in the second flow data based on the historical execution duration, and controlling the flow of the database according to the response priority.
The present invention also provides a database flow control apparatus, comprising:
the data acquisition module is used for acquiring historical monitoring data of the database; the historical monitoring data comprises historical flow data and historical load information of the database;
the load prediction module is used for carrying out iterative prediction on the load information of the database based on the historical monitoring data to obtain load prediction data of the database;
and the flow control module is used for determining a flow control strategy according to the load prediction data and controlling the flow of the database according to the flow control strategy.
The invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the database flow control method.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a database flow control method as in any one of the above.
The present invention also provides a computer program product comprising a computer program which, when executed by a processor, implements a database flow control method as described in any one of the above.
According to the database flow control method, the database flow control device, the database flow control equipment and the database flow control storage medium, iterative prediction is carried out on the load information of the database by acquiring historical monitoring data of the database, and a flow control strategy is determined according to predicted load prediction data to carry out flow control on the database. By predicting the load information of the database, the load trend of the database is predicted according to the historical flow data and the historical load information of the database, the flow control strategy is determined in advance, and the flow control is carried out on the database, so that the timeliness of the flow control is improved, the load pressure of a processor of the database is reduced, and meanwhile, the influence of sudden increase of the flow on the service is reduced by carrying out flow control in advance.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of a database flow control method provided by the present invention;
FIG. 2 is a schematic diagram of an iterative prediction process provided by the present invention;
FIG. 3 is a schematic diagram of a database flow control apparatus provided by the present invention;
fig. 4 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The database flow control method, apparatus, device and storage medium of the present invention are described below with reference to fig. 1-4.
The database flow control method provided by the invention is applied to a database, particularly a relational database such as a MySQL database, and can predict the load trend of a processor according to the flow of the database and the load information of the processor under the condition of not carrying out flow control on an upstream service system or micro service, control the database in advance and reduce the pressure of a processor of the database. Specifically, referring to fig. 1, fig. 1 is a schematic flow chart of a database flow control method according to an embodiment of the present invention, and based on fig. 1, the database flow control method according to the embodiment of the present invention includes:
firstly, historical monitoring data of a database is obtained, the historical monitoring data comprises historical flow data and historical load information of the database, and the historical load information is historical load information of a database processor (CPU). It can be understood that the load information of the database CPU is related not only to the request traffic of the database, but also to the calculation amount corresponding to each request instruction in the request traffic. Taking the query request as an example, when the query request needs a large amount of data to be queried or needs to perform joint query across multiple tables, the query request is executed slowly, so that a slow query statement is formed. Slow query statements can result in increased tasks being performed simultaneously by the database, consuming a significant amount of CPU resources.
Furthermore, the historical monitoring data is time series data, which comprises time series data of database flow and time series data of database CPU load information, and is obtained by detecting and recording the database flow and the load of the database CPU every certain time, such as 10 seconds. The detection of the database traffic and the database CPU load information may be performed synchronously or asynchronously, which is not limited herein.
200, performing iterative prediction on the load information of the database based on the historical monitoring data to obtain load prediction data of the database;
and carrying out iterative prediction on the load of the database CPU based on the acquired historical monitoring data to obtain load prediction data of the database CPU, and specifically, when the load information of the database CPU is predicted, predicting the load information of the database CPU in a future period of time based on the current load information of the database CPU and the flow data of the database. The traffic for load prediction comprises historical traffic data of a database and traffic prediction data obtained by predicting traffic based on the historical traffic data.
Further, load information of the database CPU is subjected to iterative prediction, specifically, CPU load information of a first time period in the future of the database is predicted based on historical monitoring data, CPU load information of a second time period in the future of the database is predicted based on the predicted CPU load information and the historical monitoring data, and the first time period and the second time period are continuous in time and have the same duration. According to the method, the CPU load information of the time period with the same time length in the future of the previous prediction time period is predicted based on the previous prediction result of the CPU load information, so that the CPU load information can be subjected to time continuous rolling prediction based on the previous prediction result, and the prediction data of the CPU load information of a longer time period in the future of the database can be obtained.
And determining a flow control strategy according to the load prediction data of the database, so as to control the flow of the database according to the corresponding flow control strategy. It can be understood that when the CPU load of the database is different, the traffic of the database needs to be managed to different degrees, and therefore, when the traffic is managed, different policies need to be followed. The load of the CPU of the database is predicted according to the load prediction data, so that the resource consumption required by the flow of the database can be predicted in advance, and the corresponding flow control strategy can be determined in advance to control the flow.
In this embodiment, the load information of the database is iteratively predicted by obtaining historical monitoring data of the database, and a flow control strategy is determined according to predicted load prediction data to control the flow of the database. By predicting the load information of the database, the load trend of the database is predicted according to the historical flow data and the historical load information of the database, the flow control strategy is determined in advance, and the flow control is carried out on the database, so that the timeliness of the flow control is improved, the load pressure of a processor of the database is reduced, and meanwhile, the influence of sudden increase of the flow on the service is reduced by carrying out flow control in advance.
In a preferred embodiment, the load information of the database is iteratively predicted based on a pre-trained prediction model, and the prediction model is constructed based on a deep learning neural network, such as a recurrent neural network RNN. In step 200, performing iterative prediction on load information of the database based on the historical monitoring data to obtain load prediction data of the database, which specifically includes:
step 201, taking a current time point as a reference time point, and selecting monitoring data with a first preset time length before the reference time point as prediction sample data based on the historical monitoring data;
step 202, based on the predicted sample data, predicting the monitoring data of the database in a second preset time length after the reference time point to obtain predicted monitoring data of the database;
step 203, taking the second preset time as a sliding step length, performing sliding processing on the reference time point, returning and executing the step of selecting monitoring data with a first preset time before the reference time point as prediction sample data based on the historical monitoring data, and determining load prediction data of the database with a third preset time after the current time according to the prediction monitoring data until the reference time point is a time point with a third preset time after the current time point;
the first preset time is longer than the second preset time, and the third preset time is shorter than the first preset time and longer than the second preset time.
The method includes the steps that a current time point is used as a reference time point, monitoring data with a first preset time length before the reference time point is selected as prediction sample data based on acquired historical monitoring data, wherein the first preset time length is 3 minutes for example, and the acquired historical monitoring data can be monitoring data acquired by detecting a database every 10 seconds in nearly 1 month. Selecting the monitoring data of the previous 3 minutes from the historical monitoring data as prediction sample data, predicting the monitoring data of the database to obtain the predicted monitoring data of the database after the reference time point for a second preset time length, wherein the second preset time length is less than the first preset time length, the second preset time length is 10 seconds for example, the monitoring data of the previous 3 minutes is used as a prediction basis, and predicting the monitoring data of the database after 10 seconds to obtain the predicted monitoring data of the database after the reference time point for 10 seconds.
And further, taking a second preset time length as a sliding step length, performing sliding processing on the reference time point, then re-acquiring the monitoring data of the first preset time length before the reference time point as prediction sample data, and predicting the monitoring data of the database after the reference time point for the second preset time length. That is, on the basis of predicting the monitoring data, the time point corresponding to the predicted monitoring data is used as a reference time point, and the predicted monitoring data is used as a part of predicted sample data to participate in the prediction of the monitoring data of the database in the subsequent time period, so that the iterative prediction of the monitoring data of the database is realized. Specifically, taking the above 3 minutes as a first preset time and 10 seconds as a second preset time as an example, according to the monitoring data 3 minutes before the current time point, the monitoring data 10 seconds after the current time point is predicted; predicting the monitoring data 20 seconds after the current time point according to the monitoring data 50 seconds in the first 2 minutes and the predicted monitoring data 10 seconds later; predicting the monitoring data 30 seconds after the current time point according to the data 40 seconds in the first 2 minutes and the predicted monitoring data 20 seconds later; and sequentially predicting according to the rule until the monitoring data of a third preset time length after the current time point is predicted, and determining load prediction data of the database in the third preset time length after the current time point according to the prediction monitoring data. The third preset time period is less than the first preset time period and greater than the second preset time period, that is, the third preset time period is between the first preset time period and the second preset time period, and the third preset time period is, for example, 1 minute.
Referring to a schematic process diagram of iterative prediction shown in fig. 2, a first preset time is 3 minutes, a second preset time is 10 seconds, a third preset time is 1 minute, and historical monitoring data is obtained by detecting the database once every 10 seconds. Therefore, in fig. 2, the historical monitoring data 3 minutes before the current time point includes monitoring data obtained by 18 detections on the database, the numbers in the figure represent detection orders or prediction orders, and the same number represents the monitoring data obtained by the same detection or prediction. In fig. 2, when the monitoring data in the database is predicted for the first time, the historical monitoring data obtained by the current time point reference time point and the previous 3 minutes and 18 times of detection is used as prediction sample data to predict the monitoring data 10 seconds after the database; then, performing sliding processing on the reference time point, taking the time point 10 seconds after the current time point as the reference time point, taking the historical monitoring data obtained by the previous 17 detections and the monitoring data predicted for the first time as prediction sample data, performing second prediction on the monitoring data of the database, and predicting the monitoring data of the database 10 seconds after the reference time point; according to the method, iterative prediction is carried out on the monitoring data of the database in a sliding window processing mode, and after 6 times of prediction is completed, the predicted monitoring data of the database 1 minute after the current time point is obtained.
Further, the forecast sample data includes traffic samples and load samples, and the forecast monitoring data includes forecast data of traffic and forecast data of load information of the database, that is, the first forecast traffic and the first load forecast data. In step 202, based on the predicted sample data, predicting the monitoring data of the database for a second preset time after the reference time point to obtain predicted monitoring data of the database, which further includes:
2021, predicting the flow of the database for a second preset time after the reference time point based on the flow sample in the predicted sample data to obtain a first predicted flow;
step 2022, predicting load information of the database in a second preset time after the reference time point according to the first predicted flow and the load sample in the predicted sample data to obtain first load predicted data.
Firstly, based on a flow sample in the prediction sample data, predicting the flow of the database for a second preset time length after the reference time point to obtain a first predicted flow of the database for the second preset time length after the reference time point. When the flow prediction is carried out on the database, data fitting is carried out on flow samples, the flow change trend is predicted, and the flow of the database after a period of time is predicted by combining the flow change trend prediction based on the current flow of the database.
And predicting load information of the database for a second preset time length after the reference time point according to the predicted flow data and the load sample in the predicted sample data, so as to obtain first load prediction data of the database for the second preset time length after the reference time point. When load information of a second preset time after a reference time point of the database is predicted, data fitting is specifically carried out on a load sample to predict load change trend, on the basis, the predicted load change trend is adjusted based on the flow change trend and/or the first predicted flow, and load information of the second preset time after the reference time point of the database is predicted according to the adjusted load change trend to obtain first load prediction data.
Therefore, based on historical monitoring data, monitoring data of a certain time length in the future can be predicted, if the predicted time length is short, the requirement for timeliness of flow pipe control cannot be met, and if the predicted time length is long, the accuracy of the predicted data is difficult to guarantee. Monitoring data in a shorter time in the future is predicted based on historical monitoring data in an iterative prediction mode, so that the prediction time is shortened, and the accuracy of the predicted data is guaranteed; on the basis of the predicted monitoring data, the subsequent monitoring data are continuously predicted, the predicted monitoring data in a longer time period in the future can be obtained, and on the basis of ensuring the accuracy of the predicted data, the demand on the timeliness of flow pipe control is met.
In this embodiment, by performing iterative prediction on the monitoring data in the database, the prediction duration can be shortened, which is beneficial to ensuring the accuracy of the prediction data, and the predicted monitoring data in a longer time period in the future can be obtained, so that the demand on the timeliness of the flow pipe control is met on the basis of ensuring the accuracy of the prediction data.
In a preferred embodiment, the flow control policy includes at least one of a slow query termination policy, a slow query filtering policy and a priority policy, and different policies may be used alone or in combination, specifically, in step 300, the determining the flow control policy according to the load prediction data includes:
step 301, determining a load gradient interval corresponding to the load prediction data;
step 302, determining a flow control strategy according to the load gradient interval; wherein; the load gradient interval comprises a plurality of load gradient intervals, and different load gradient intervals correspond to different flow control strategies.
When the flow control strategy is determined, a load gradient interval corresponding to load prediction data is determined, and the flow control strategy is determined according to the load gradient interval. In this embodiment, the CPU load of the database is divided into multiple gradient sections, each gradient section corresponds to a CPU load range, different gradient sections correspond to different flow control strategies, and it is determined in which gradient section the predicted CPU load of the database is located according to the load prediction data, where the flow control strategy corresponding to the gradient section is the strategy that is finally used for controlling the flow of the database.
It should be noted that, the prediction of the monitoring data of the database is dynamically iterative, and therefore, the flow control policy is also dynamically changed, specifically, the flow control policy is dynamically adjusted according to the latest load prediction data as time passes. Different flow control strategies have different flow control modes, specifically, if the flow control strategy includes a slow query termination strategy, in step 300, performing flow control on the database according to the flow control strategy includes:
step 303, acquiring current target flow data of the database; the target flow data comprises first flow data with an execution state being execution and second flow data with an execution state being to be executed;
step 304, identifying a first slow query statement from the first flow data, wherein the execution duration of the first slow query statement exceeds a fourth preset duration;
step 305, executing a termination operation on the first slow query statement to perform flow control on the database.
When the flow control is performed on the database, current target flow data of the database is obtained first, wherein the target flow data comprises first flow data with an execution state being execution and second flow data with the execution state being to be executed. And for the first flow data with the execution state being in execution, identifying a slow query statement with the execution duration exceeding a fourth preset duration, executing termination operation on the slow query statement, and releasing the CPU load of the database. Wherein the fourth preset time period is, for example, 3 seconds.
If the flow control policy includes a slow query filtering policy, in step 300, performing flow control on the database according to the flow control policy, which may further include:
step 306, identifying a second slow query statement from the second flow data;
step 307, filtering the second traffic data based on the second slow query statement to flow control the database.
If the flow control strategy comprises a slow query filtering strategy, identifying a slow query statement to be executed from second flow data to be executed, filtering the second flow data to be executed based on the slow query statement, filtering the identified slow query statement from the second flow data, and controlling the flow of the database according to the filtered slow query statement. When the slow query statement is identified, the execution duration of each query statement can be determined based on the historical monitoring data, the query statement with the execution duration exceeding a fourth preset duration is the slow query statement, and the query statement to be executed is matched with the identified slow query statement based on the type of the query statement to be executed, the service type corresponding to the query statement and the like, so that the slow query statement in the second flow data is identified.
Further, if the flow control policy includes a priority policy, in step 300, performing flow control on the database according to the flow control policy, which may further include:
step 308, determining the historical execution duration of each query statement in the second flow data according to the historical monitoring data;
step 309, determining the response priority of each query statement in the second flow data based on the historical execution duration, and controlling the flow of the database according to the response priority.
If the flow control strategy comprises a priority strategy, determining the historical execution time of each query statement in the second flow data according to the historical flow data in the historical monitoring data, wherein the historical execution time can be the average value of multiple execution times in the historical record. And determining the response priority of each query statement in the second flow data based on the historical execution duration of each query statement, and controlling the flow of the database according to the response priority. Generally, when the priorities of query statements are set, for the same type of query statements, the shorter the execution time, the higher the priority, and under the same load, the larger the number of requests that can be concurrently processed, so that traffic congestion can be reduced.
Further, the slow query termination policy, the slow query filtering policy, and the priority policy may be used individually or in combination, for example, if the flow control policy includes the slow query filtering policy and the priority policy, for the second traffic data to be executed, the slow query statements may be filtered based on the slow query filtering policy, then the response priority of each query statement in the filtered traffic data may be determined based on the priority policy, and the database may be subjected to flow control according to the response priority. The priority of the filtered slow query statement may be the lowest priority or the highest priority, and is specifically determined according to the load prediction data, which is not specifically limited herein.
In this embodiment, the load of the database is subjected to gradient division, and different flow control strategies are set for different load gradient intervals, so that the flow control of the database in different degrees can be realized according to predicted load information, the flexibility of flow control is improved, and further, the influence on the service is reduced as much as possible on the premise that the load pressure of the database is ensured to normally provide the service.
The database flow control apparatus provided by the present invention is described below, and the database flow control apparatus described below and the database flow control method described above may be referred to in correspondence with each other.
Referring to fig. 3, a database flow control apparatus according to an embodiment of the present invention includes:
the data acquisition module 10 is used for acquiring historical monitoring data of a database; the historical monitoring data comprises historical flow data and historical load information of the database;
the load prediction module 20 is configured to perform iterative prediction on the load information of the database based on the historical monitoring data to obtain load prediction data of the database;
and the flow control module 30 is configured to determine a flow control policy according to the load prediction data, and perform flow control on the database according to the flow control policy.
In one embodiment, the load prediction module 20 is further configured to:
selecting monitoring data with a first preset time length before the reference time point as prediction sample data based on the historical monitoring data by taking the current time point as the reference time point;
predicting the monitoring data of the database for a second preset time after the reference time point based on the predicted sample data to obtain predicted monitoring data of the database;
taking the second preset time as a sliding step length, performing sliding processing on the reference time point, returning and executing the step of selecting monitoring data with a first preset time before the reference time as prediction sample data based on the historical monitoring data, and determining load prediction data of the database with a third preset time after the current time according to the prediction monitoring data until the reference time is a time after the current time;
the first preset time is longer than the second preset time, and the third preset time is shorter than the first preset time and longer than the second preset time.
In one embodiment, the prediction sample data comprises traffic samples and load samples; the predictive monitoring data comprises first predicted traffic and first load predictive data; the load prediction module 20 is further configured to:
predicting the flow of the database for a second preset time after the reference time point based on the flow sample in the predicted sample data to obtain a first predicted flow;
and predicting load information of the database in a second preset time after the reference time point according to the first predicted flow and the load sample in the predicted sample data to obtain first load predicted data.
In one embodiment, the flow control module 30 is further configured to:
determining a load gradient interval corresponding to the load prediction data;
determining a flow control strategy according to the load gradient interval; wherein; the load gradient interval comprises a plurality of load gradient intervals, and different load gradient intervals correspond to different flow control strategies.
In one embodiment, the flow control policy comprises at least one of a slow query termination policy, a slow query filtering policy, and a priority policy; if the flow control policy includes the slow query termination policy, the flow control module 30 is further configured to:
acquiring current target flow data of the database; the target flow data comprises first flow data with an execution state being execution and second flow data with an execution state being to be executed;
identifying a first slow query statement with execution duration exceeding a fourth preset duration from the first flow data;
and executing a termination operation on the first slow query statement to control the flow of the database.
In one embodiment, if the flow control policy includes the slow query filtering policy, the flow control module 30 is further configured to:
identifying a second slow query statement from the second traffic data;
filtering the second traffic data based on the second slow query statement to flow control the database.
In an embodiment, if the flow control policy includes the priority policy, the flow control module 30 is further configured to:
determining the historical execution duration of each query statement in the second flow data according to the historical monitoring data;
and determining the response priority of each query statement in the second flow data based on the historical execution duration, and controlling the flow of the database according to the response priority.
Fig. 4 illustrates a physical structure diagram of an electronic device, which may include, as shown in fig. 4: a processor (processor) 410, a communication Interface 420, a memory (memory) 430 and a communication bus 440, wherein the processor 410, the communication Interface 420 and the memory 430 are communicated with each other via the communication bus 440. Processor 410 may invoke logic instructions in memory 430 to perform a database flow control method comprising:
acquiring a query statement of a database, and a query plan and a database appearance corresponding to the query statement;
determining a target query channel corresponding to the query statement according to the query plan;
if the target query channel is a first query channel, constructing a bloom filter based on the database appearance;
and executing the query plan based on the bloom filter to obtain target data corresponding to the query statement.
In addition, the logic instructions in the memory 430 may be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, where the computer program product includes a computer program, the computer program may be stored on a non-transitory computer readable storage medium, and when the computer program is executed by a processor, a computer can execute the database flow control method provided by the above methods, where the method includes:
acquiring historical monitoring data of a database; the historical monitoring data comprises historical flow data and historical load information of the database;
carrying out iterative prediction on the load information of the database based on the historical monitoring data to obtain load prediction data of the database;
and determining a flow control strategy according to the load prediction data, and controlling the flow of the database according to the flow control strategy.
In yet another aspect, the present invention also provides a non-transitory computer readable storage medium, on which a computer program is stored, the computer program being implemented by a processor to perform the database flow control method provided by the above methods, the method including:
acquiring historical monitoring data of a database; the historical monitoring data comprises historical flow data and historical load information of the database;
carrying out iterative prediction on the load information of the database based on the historical monitoring data to obtain load prediction data of the database;
and determining a flow control strategy according to the load prediction data, and controlling the flow of the database according to the flow control strategy.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A method for controlling database traffic, the method comprising:
acquiring historical monitoring data of a database; the historical monitoring data comprises historical flow data and historical load information of the database;
carrying out iterative prediction on the load information of the database based on the historical monitoring data to obtain load prediction data of the database;
and determining a flow control strategy according to the load prediction data, and controlling the flow of the database according to the flow control strategy.
2. The method according to claim 1, wherein iteratively predicting the load information of the database based on the historical monitoring data to obtain load prediction data of the database includes:
selecting monitoring data with a first preset time length before the reference time point as prediction sample data based on the historical monitoring data by taking the current time point as the reference time point;
predicting the monitoring data of the database for a second preset time after the reference time point based on the predicted sample data to obtain predicted monitoring data of the database;
taking the second preset time as a sliding step length, performing sliding processing on the reference time point, returning and executing the step of selecting monitoring data with a first preset time before the reference time as prediction sample data based on the historical monitoring data, and determining load prediction data of the database with a third preset time after the current time according to the prediction monitoring data until the reference time is a time after the current time;
the first preset time is longer than the second preset time, and the third preset time is shorter than the first preset time and longer than the second preset time.
3. The database flow control method according to claim 2, wherein the prediction sample data includes a flow sample and a load sample; the predictive monitoring data comprises first predicted traffic and first load predictive data; the predicting the monitoring data of the database for a second preset time after the reference time point based on the prediction sample data to obtain the predicted monitoring data of the database comprises:
predicting the flow of the database for a second preset time after the reference time point based on the flow sample in the predicted sample data to obtain a first predicted flow;
and predicting load information of the database in a second preset time after the reference time point according to the first predicted flow and the load sample in the predicted sample data to obtain first load predicted data.
4. The method according to claim 1, wherein said determining a flow control policy based on said load forecast data comprises:
determining a load gradient interval corresponding to the load prediction data;
determining a flow control strategy according to the load gradient interval; wherein; the load gradient interval comprises a plurality of load gradient intervals, and different load gradient intervals correspond to different flow control strategies.
5. The database flow control method according to claim 1, wherein the flow control policy comprises at least one of a slow query termination policy, a slow query filtering policy, and a priority policy; if the flow control policy includes the slow query termination policy, the performing flow control on the database according to the flow control policy includes:
acquiring current target flow data of the database; the target flow data comprises first flow data with an execution state being execution and second flow data with an execution state being to be executed;
identifying a first slow query statement with execution duration exceeding a fourth preset duration from the first flow data;
and executing a termination operation on the first slow query statement to control the flow of the database.
6. The method of claim 5, wherein if the flow control policy includes the slow query filtering policy, the controlling the flow of the database according to the flow control policy comprises:
identifying a second slow query statement from the second traffic data;
filtering the second traffic data based on the second slow query statement to flow control the database.
7. The method of claim 5, wherein if the traffic control policy includes the priority policy, the performing traffic control on the database according to the traffic control policy comprises:
determining the historical execution duration of each query statement in the second flow data according to the historical monitoring data;
and determining the response priority of each query statement in the second flow data based on the historical execution duration, and controlling the flow of the database according to the response priority.
8. A database flow control apparatus, comprising:
the data acquisition module is used for acquiring historical monitoring data of the database; the historical monitoring data comprises historical flow data and historical load information of the database;
the load prediction module is used for carrying out iterative prediction on the load information of the database based on the historical monitoring data to obtain load prediction data of the database;
and the flow control module is used for determining a flow control strategy according to the load prediction data and controlling the flow of the database according to the flow control strategy.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the database flow control method according to any of claims 1 to 7 when executing the program.
10. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements the database flow control method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211680276.XA CN115658701B (en) | 2022-12-27 | 2022-12-27 | Database flow control method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211680276.XA CN115658701B (en) | 2022-12-27 | 2022-12-27 | Database flow control method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115658701A true CN115658701A (en) | 2023-01-31 |
| CN115658701B CN115658701B (en) | 2023-03-14 |
Family
ID=85023136
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211680276.XA Active CN115658701B (en) | 2022-12-27 | 2022-12-27 | Database flow control method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115658701B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014038737A1 (en) * | 2012-09-07 | 2014-03-13 | 에스케이텔레콤 주식회사 | Network traffic management system using monitoring policy and filtering policy, and method thereof |
| CN113986564A (en) * | 2021-09-22 | 2022-01-28 | 鸬鹚科技(深圳)有限公司 | Application data flow monitoring method and device, computer equipment and medium |
| WO2022068540A1 (en) * | 2020-09-30 | 2022-04-07 | 深圳前海微众银行股份有限公司 | Database status determination method and apparatus, device, and storage medium |
-
2022
- 2022-12-27 CN CN202211680276.XA patent/CN115658701B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014038737A1 (en) * | 2012-09-07 | 2014-03-13 | 에스케이텔레콤 주식회사 | Network traffic management system using monitoring policy and filtering policy, and method thereof |
| WO2022068540A1 (en) * | 2020-09-30 | 2022-04-07 | 深圳前海微众银行股份有限公司 | Database status determination method and apparatus, device, and storage medium |
| CN113986564A (en) * | 2021-09-22 | 2022-01-28 | 鸬鹚科技(深圳)有限公司 | Application data flow monitoring method and device, computer equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115658701B (en) | 2023-03-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| AU2021232839A1 (en) | Updating Attribute Data Structures to Indicate Trends in Attribute Data Provided to Automated Modelling Systems | |
| CN109960573B (en) | Cross-domain computing task scheduling method and system based on intelligent perception | |
| TWI738721B (en) | Task scheduling method and device | |
| CN108595254B (en) | Query scheduling method | |
| US20210255899A1 (en) | Method for Establishing System Resource Prediction and Resource Management Model Through Multi-layer Correlations | |
| US20190347621A1 (en) | Predicting task durations | |
| CN106168912A (en) | The big data platform of a kind of Hadoop runs the dispatching method of time Estimate based on backup tasks | |
| CN113128787A (en) | Telephone traffic prediction method, system, device and medium | |
| US20200050982A1 (en) | Method and System for Predictive Modeling for Dynamically Scheduling Resource Allocation | |
| CN114170002A (en) | Method and device for predicting access frequency | |
| CN114301803B (en) | Network quality detection method and device, electronic equipment and storage medium | |
| CN115658701B (en) | Database flow control method, device, equipment and storage medium | |
| US11468348B1 (en) | Causal analysis system | |
| CN110413482B (en) | Detection method and device | |
| CN116107854A (en) | Method, system, equipment and medium for predicting operation maintenance index of computer | |
| CN115330219A (en) | Resource scheduling method and device | |
| CN114117785A (en) | Situation awareness realization method, device, equipment and medium | |
| CN115484143A (en) | Alarm processing method and device, electronic equipment and storage medium | |
| CN114355793A (en) | Training method and device of automatic driving planning model for vehicle simulation evaluation | |
| CN113723692A (en) | Data processing method, apparatus, device, medium, and program product | |
| CN113129127A (en) | Early warning method and device | |
| CN111340648A (en) | Energy management and control method and system based on Internet of things | |
| CN113837782A (en) | Method and device for optimizing periodic item parameters of time series model and computer equipment | |
| CN111625352A (en) | Scheduling method, device and storage medium | |
| CN111190800A (en) | Method, system, device and storage medium for predicting batch operation duration of host |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |