CN116401236A - Method and equipment for adaptively optimizing database parameters - Google Patents
Method and equipment for adaptively optimizing database parameters Download PDFInfo
- Publication number
- CN116401236A CN116401236A CN202310665655.XA CN202310665655A CN116401236A CN 116401236 A CN116401236 A CN 116401236A CN 202310665655 A CN202310665655 A CN 202310665655A CN 116401236 A CN116401236 A CN 116401236A
- Authority
- CN
- China
- Prior art keywords
- database system
- information
- performance data
- data
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000005457 optimization Methods 0.000 claims abstract description 93
- 238000012545 processing Methods 0.000 claims abstract description 57
- 238000012549 training Methods 0.000 claims abstract description 42
- 230000004044 response Effects 0.000 claims abstract description 31
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 20
- 230000002068 genetic effect Effects 0.000 claims abstract description 19
- 230000001960 triggered effect Effects 0.000 claims abstract description 11
- 238000007781 pre-processing Methods 0.000 claims abstract description 10
- 238000011156 evaluation Methods 0.000 claims description 26
- 230000035772 mutation Effects 0.000 claims description 18
- 238000012544 monitoring process Methods 0.000 claims description 15
- 230000003044 adaptive effect Effects 0.000 claims description 13
- 230000008569 process Effects 0.000 claims description 13
- 238000000926 separation method Methods 0.000 claims description 11
- 230000006870 function Effects 0.000 claims description 9
- 238000012360 testing method Methods 0.000 claims description 8
- 238000013515 script Methods 0.000 claims description 7
- 238000003860 storage Methods 0.000 claims description 4
- 238000004140 cleaning Methods 0.000 claims description 3
- 238000013500 data storage Methods 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 2
- 230000008859 change Effects 0.000 description 6
- 238000004458 analytical method Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 3
- 230000011218 segmentation Effects 0.000 description 3
- 239000000872 buffer Substances 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010205 computational analysis Methods 0.000 description 1
- 238000004883 computer application Methods 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/217—Database tuning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/12—Computing arrangements based on biological models using genetic models
- G06N3/126—Evolutionary algorithms, e.g. genetic algorithms or genetic programming
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Biophysics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Biomedical Technology (AREA)
- Genetics & Genomics (AREA)
- Artificial Intelligence (AREA)
- Physiology (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the specification discloses a method and equipment for self-adaptive optimization of database parameters, which are applied to the technical field of data processing and are used for solving the problems of high professional and low efficiency of manual configuration of a database. The method comprises the following steps: collecting basic configuration information of server hardware, and acquiring processing event response information of a server corresponding to a database system in a preset time period to judge whether the basic configuration information triggers an optimization thread; if the optimization thread is triggered, collecting performance data of a database system; preprocessing the performance data to obtain training data, and training an initial supervised learning model based on the training data to obtain a parameter prediction model; inputting each configuration parameter of the database system and a preset value range thereof into a parameter prediction model to output initial configuration parameters; and iterating the initial configuration parameters based on a preset genetic optimization algorithm to obtain the optimal configuration parameters corresponding to the database system so as to perform optimization recommendation.
Description
Technical Field
The present disclosure relates to the field of database optimization technologies, and in particular, to a method and apparatus for adaptive optimization of database parameters.
Background
Database systems are critical components of modern computer systems, having a significant impact on the performance and efficiency of applications, most of which have many configurable parameters to control and affect the performance of applications. With the rapid development of computer applications and network technologies and continuous improvement of architecture, the demands for database systems are becoming wider and wider, so that in the production process, if a database processing bottleneck is encountered to ensure the processing efficiency of the applications, the database parameters need to be allocated, so that the optimization of the database parameters is an important technical means for ensuring the service quality of the application programs in the database systems.
The optimization of configuration parameters for database systems in the traditional manner is manually configured by a skilled practitioner based on experience. However, the manual configuration process is not known and set correctly for a normal user in a short time, but based on the process of configuration by technicians, the client site hardware equipment information needs to be docked, the server hardware information and database performance data are collected through pressure test and busy hour service, and the database parameters are reconfigured after analysis. In this case, firstly, the configuration time becomes long, so that the database before configuration cannot perform the optimal performance in the production and use process, and the processing business is blocked or delayed, and secondly, the cost of the configuration mode based on technicians is high.
Disclosure of Invention
To solve the above technical problems, one or more embodiments of the present disclosure provide a method and apparatus for adaptive optimization of database parameters.
One or more embodiments of the present disclosure adopt the following technical solutions:
one or more embodiments of the present specification provide a method for adaptive optimization of database parameters, the method comprising:
collecting basic configuration information of server hardware based on an information acquisition command, and acquiring processing event response information of a corresponding database system of the server within a preset time period;
judging whether the database system triggers an optimization thread or not based on the basic configuration information and the processing event response information;
if the optimization thread is triggered, collecting performance data of the database system; wherein the performance data comprises: performance index and workload parameters;
preprocessing the performance data to obtain training data, and training an initial supervised learning model based on the training data to obtain a parameter prediction model;
inputting each configuration parameter of the database system and a preset value range thereof into the parameter prediction model to output initial configuration parameters;
And iterating the initial configuration parameters based on a preset genetic optimization algorithm to obtain the optimal configuration parameters corresponding to the database system so as to perform optimization recommendation.
Optionally, in one or more embodiments of the present disclosure, the iterating the initial configuration parameters based on a preset genetic optimization algorithm to obtain optimal configuration parameters corresponding to the database system for optimization recommendation specifically includes:
randomly generating a preset number of configuration parameter combinations in a preset value range of each configuration parameter to obtain an initialization population; wherein each of the configuration parameters are combined into an individual in the initialized population;
acquiring performance indexes corresponding to the configuration parameter combinations, and taking the corresponding performance indexes as fitness functions of the initialized population;
determining parent individuals of the initialized population based on a roulette selection mode, so as to perform cross operation on the parent individuals based on single-point cross and obtain child individuals of the initialized population;
performing mutation operation on the child individuals according to a random mutation mode to obtain variant child individuals of the initialized population, so as to determine fitness values corresponding to the variant child individuals based on the fitness function;
If the fitness value is smaller than a preset fitness value, forming a current population based on the child individuals, the variant child individuals and a preset number of parent individuals, so as to obtain a configuration parameter combination corresponding to the optimal individual by performing iterative operation on the current population;
and combining the configuration parameters corresponding to the optimal individuals to serve as optimal configuration parameters corresponding to the database system, so that the configuration parameters of the database system are updated and replaced based on the optimal configuration parameters.
Optionally, in one or more embodiments of the present disclosure, before determining whether the database system triggers an optimization thread based on the basic configuration information and the processing event response information, the method further includes:
storing the basic configuration information in a designated configuration file based on a preset first format; wherein the basic configuration information includes: CPU information, memory information, hard disk information and network card information; the preset first format is used for defining separation parameters of the storage process of the basic configuration information; the separation parameters include: a separator and a separation width;
setting corresponding scripts based on initialization suggestions of preset parameters so as to determine initialization parameter values of configuration parameters in the database system based on the scripts; wherein the initialization suggestion is set corresponding to the basic configuration information;
And initializing the database system based on the initialization parameter value, and restarting the initialized database.
Optionally, in one or more embodiments of the present specification, collecting performance data of the database system specifically includes:
collecting performance data in a unit time period of the database system based on a preset command of the database system, and storing the performance data in a designated performance file based on a preset second format; wherein the preset second format is used for defining a separator and a separation width of each performance data storage process.
Optionally, in one or more embodiments of the present disclosure, based on a preset command of the database system, collecting performance data in a unit time period of the database system specifically includes:
collecting performance indexes of the database system according to a built-in monitoring tool or a third-party tool of the database system;
acquiring a log file of the database system to extract the workload information of the database system in the log file; wherein the workload information includes: query type, time consuming query, number of concurrent connections;
And extracting the performance index and the workload information within a preset unit time period as performance data.
Optionally, in one or more embodiments of the present disclosure, the collecting, according to a built-in monitoring tool or a third party tool of the database system, a performance index of the database system specifically includes:
based on a first collection command of the built-in monitoring tool or the third party tool, obtaining the CPU utilization rate of the database system, and based on a preset unitization command, obtaining an average value of the CPU utilization rate in a unit time period; and
acquiring the memory utilization rate of the database system in a unit time period according to the second collection command of the built-in monitoring tool or the third party tool; and
acquiring read-write information of the database system according to a third collection command of the built-in monitoring tool or the third party tool so as to acquire disk network throughput in a unit time period of the database system based on the read-write information; wherein the read-write information includes: a read-write rate per second, a read-write delay;
and taking the average value of the CPU utilization rate in the unit time period, the memory utilization rate of the database system in the unit time period and the disk network throughput in the unit time period as performance indexes of the database system.
Optionally, in one or more embodiments of the present disclosure, preprocessing the performance data to obtain training data, so as to train an initial supervised learning model based on the training data, to obtain a parameter prediction model, specifically including:
performing data cleaning on the performance data to obtain filtered performance data so as to extract key performance data of the filtered performance data;
based on the input size of the initial supervised learning model, performing data scaling on the key performance data to obtain training data;
dividing the training data into a training set and a testing set, so as to train the initial supervised learning model based on the training set and obtain a trained supervised learning model;
and if the trained supervised learning model meets the requirements based on the test set, taking the trained supervised learning model as a parameter prediction model meeting the requirements.
Optionally, in one or more embodiments of the present disclosure, extracting the key performance data of the filtered performance data specifically includes:
acquiring a plurality of associated computing nodes connected in series with the computing node where the database system is located;
acquiring historical configuration parameter adjustment records of the database system and the database system of the associated computing node;
Clustering the performance data in the historical configuration parameter adjustment record based on data characteristics to obtain clustering sets of the performance data, and determining first key performance data based on the number of the performance data in each clustering set; wherein the data features are used to characterize the influencing object of the performance data;
acquiring historical performance data of a plurality of historical unit time periods, and comparing the historical performance data with the performance data in pairs based on time sequence to obtain a differential data set; wherein the historical unit time period corresponds to the unit time period;
obtaining mutation data of each performance data in the differential data set, and taking the performance data corresponding to the mutation data as second key performance data if the numerical value of the mutation data is larger than the preset variation frequency of the performance data;
and determining the key performance data of the filtered performance data according to the union set of the first key performance data and the second key performance data.
Optionally, in one or more embodiments of the present disclosure, determining whether the database system triggers an optimization thread based on the basic configuration information and the processing event response information specifically includes:
Acquiring current component configuration data of the server hardware in the basic configuration information, and triggering the optimization thread if the current component configuration data does not exist in a preset component configuration data range;
if the current component configuration data exists in the preset component configuration data range, determining that the computing node depends on an application program of the database system;
acquiring the work order processing information and the application evaluation information uploaded in the preset time period of each application program, and taking the work order processing information and the application evaluation information as processing event response information;
splitting the work order processing information based on a preset word splitting length to obtain a plurality of word splitting words, and filtering connective word words in the work order processing information based on semantic attributes of the word splitting words to obtain keywords to be identified of the work order processing information;
acquiring preset description information corresponding to each configuration parameter, determining the number of the configuration parameters corresponding to the work order processing information based on the matching relation between the keywords to be identified and the preset description information, and determining a first optimized weight value based on the number of the configuration parameters corresponding to the work order processing information;
Based on the evaluation value of the application evaluation information, extracting corresponding negative evaluation information to preprocess the negative evaluation information and obtain keywords to be analyzed associated with the basic configuration information;
determining a second optimized weight value based on the ratio of the negative evaluation information of the keywords to be analyzed to all the negative evaluation information and the number of the keywords to be analyzed;
and determining the weight value of the basic configuration information triggering optimization thread based on the first optimization weight value and the second optimization weight value, wherein the weight value is larger than a preset weight value and triggers the optimization thread.
Optionally, in one or more embodiments of the present disclosure, after iterating the initial configuration parameters based on a preset genetic optimization algorithm to obtain optimal configuration parameters corresponding to the database system for optimization recommendation, the method further includes:
acquiring first model data of each hardware configuration in basic configuration information of the associated computing node, and acquiring second model data of each hardware configuration in the basic configuration information of the computing node where the database system is located;
if the first model data is consistent with the second model data, synchronizing the optimal configuration parameters and performance data of the database system corresponding to the optimal configuration parameters to the associated computing node;
And if the performance data of the associated computing node is matched with the performance data of the database system corresponding to the optimal configuration parameter, optimizing the database system of the associated computing node based on the optimal configuration parameter.
One or more embodiments of the present specification provide an apparatus for adaptive optimization of database parameters, the apparatus comprising:
at least one processor; the method comprises the steps of,
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores instructions executable by the at least one processor to enable the at least one processor to:
collecting basic configuration information of server hardware based on an information acquisition command, and acquiring processing event response information of a corresponding database system of the server within a preset time period;
judging whether the database system triggers an optimization thread or not based on the basic configuration information and the processing event response information;
if the optimization thread is triggered, collecting performance data of the database system; wherein the performance data comprises: performance index and workload parameters;
Preprocessing the performance data to obtain training data, and training an initial supervised learning model based on the training data to obtain a parameter prediction model;
inputting each configuration parameter of the database system and a preset value range thereof into the parameter prediction model to output initial configuration parameters;
and iterating the initial configuration parameters based on a preset genetic optimization algorithm to obtain the optimal configuration parameters corresponding to the database system so as to perform optimization recommendation.
The above-mentioned at least one technical scheme that this description embodiment adopted can reach following beneficial effect:
whether the database system triggers the optimization thread is judged through the basic configuration information and the processing event response information, so that updating optimization of the database system based on timely triggering of server performance is realized. The parameters of the database system are predicted by combining a genetic optimization algorithm after the parameter prediction model is obtained according to the database performance data training, and the optimal configuration parameters are determined to optimize the parameters of the database system, so that the problems of high labor cost and low optimization efficiency caused by the fact that support personnel in the prior art need to dock client site hardware equipment information, collect server hardware information and database performance data through pressure test and busy hour service, and then analyze and reconfigure the database parameters are solved.
Drawings
In order to more clearly illustrate the embodiments of the present description or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some of the embodiments described in the present description, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. In the drawings:
FIG. 1 is a schematic flow chart of a method for adaptive optimization of database parameters according to an embodiment of the present disclosure;
FIG. 2 is a schematic logic flow diagram of adaptive optimization of database parameters according to an embodiment of the present disclosure;
fig. 3 is a schematic diagram of an internal structure of a device for adaptive optimization of database parameters according to an embodiment of the present disclosure.
Detailed Description
The embodiment of the specification provides a method and equipment for adaptively optimizing database parameters.
In order to make the technical solutions in the present specification better understood by those skilled in the art, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is obvious that the described embodiments are only some embodiments of the present specification, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, shall fall within the scope of the present disclosure.
As shown in fig. 1, in one or more embodiments of the present disclosure, a method flow diagram for adaptive optimization of database parameters is provided. As can be seen from fig. 1, in one or more embodiments of the present disclosure, a method for adaptively optimizing database parameters includes the following steps:
s101: and collecting basic configuration information of server hardware based on the information acquisition command, and acquiring processing event response information of the server corresponding to a preset time period of a database system.
Because the performance of the database directly affects the service quality of the application program in the database system, the database system needs to be optimally updated in time so that the application operation can be supported by the database. In order to realize the intelligent automatic judgment of whether the database needs to be subjected to parameter optimization, the problems that extra labor force is required for human analysis and the performance condition of the database is difficult to grasp in time by the human analysis are solved. In the embodiment of the present specification, the performance of the database system depends on hardware of the server, for example: the network throughput of the server depends on the disk information in the hardware. Therefore, in the embodiment of the present disclosure, the basic configuration information of the server is collected according to the information acquisition command, and then, in order to comprehensively consider the requirement of personnel on database update, the database update can be performed in time. In the embodiment of the present disclosure, processing time response information of a database system corresponding to a server in a preset time period is obtained. Wherein, it should be noted that the basic configuration information includes: CPU information, memory information, hard disk information and network card information, so it can be understood that different types of information in the basic configuration information need different information acquisition commands to acquire the information. For example: CPU information is information such as CPU model, core number, thread number, frequency and the like obtained by a command lscpu; the memory information is information such as total memory, used memory, available memory and the like obtained through a command free-m; the hard disk information is information such as a hard disk name, a partition, a size and the like obtained through a command lsblk; the network card information is information such as a network card name, an IP address, an MAC address and the like obtained through the command ipaddrshow. Further, it should be further noted that the processing event response information includes work order processing information and application evaluation information, that is, update requirements of each application for the database may be determined based on the processing event response information. Data support is provided for subsequent decisions on database updates by collection of multidimensional data.
S102: and judging whether the database system triggers an optimization thread or not based on the basic configuration information and the processing event response information.
After the basic configuration information of the server hardware and the processing event response information of the database system are obtained based on the step S101, whether to trigger the optimization thread of the database system can be judged based on the multidimensional data, so that timely update optimization of the teammate database system can be realized in response to the requirements of users and applications.
Specifically, in one or more embodiments of the present disclosure, determining whether the basic configuration information triggers an optimization thread based on the basic configuration information and the processing event response information specifically includes the following steps:
firstly, current component configuration data of server hardware in basic configuration information is obtained, and if the current component configuration data does not exist in the preset component configuration data range, the optimization thread is triggered. Based on the above step S101, it can be known that the basic configuration information includes CPU information, memory information, hard disk information and network card information, and the current component configuration data is data forming the basic configuration information, for example, taking the basic configuration information as the CPU information, the current component configuration data includes data such as CPU model, core number, thread number, frequency, etc. At this time, if it is determined that the current component configuration data does not exist in the preset component configuration data range, it is difficult for the current server hardware to support the configuration of the current database system, so in order to ensure the performance of the database system and the normal use of the database system, the optimization thread of the database system needs to be triggered to perform the subsequent optimization operation.
If the current component configuration data exists within the preset component configuration data, then it is determined that the computing node where the current database system is located depends on the application of the database system. And then acquiring the work order processing information and the application evaluation information uploaded in the preset time period of each application program, so that the work order processing information and the application evaluation information are used as processing event response information, and the problems and the evaluation information of a system fed back by each user are used as processing event response information for analyzing whether the database system is optimized or not. In order to effectively analyze the response information of the processing event, the processing information of the work order is split according to the preset word segmentation length to obtain a plurality of word segmentation words, and then the word of the connecting word in the processing information of the work order is filtered according to the semantic attribute of each word segmentation word to obtain the key word to be identified of the processing information of the work order. And acquiring preset description information corresponding to each configuration parameter in the database system, determining the number of the configuration parameters in the work order processing information based on the matching relation between the keywords to be identified and the preset description information, and determining a first optimized weight value according to the number of the configuration parameters corresponding to the work order processing information. For example: the more the number of configuration parameters corresponding to the work order processing information, the more the database system needs to be optimized, and the higher the first optimization weight value is. And then extracting the corresponding negative evaluation information according to the evaluation value of the application evaluation information, so as to preprocess the negative evaluation information and obtain the keywords to be analyzed which are related to the basic configuration information. And then calculating to obtain the ratio of the negative evaluation information of the keywords to be analyzed to all the negative evaluation information and the number of the keywords to be analyzed, thereby determining a second optimized weight value according to the determined ratio and number. And determining the weight value of the basic configuration information triggering the optimizing thread according to the first optimizing weight value and the second optimizing weight value, and triggering the corresponding optimizing thread if the weight value is larger than the preset weight value.
Further, since, in addition to optimization of database system parameters, when a user newly installs the database management system, for a large number of complex configuration parameters in the database configuration file, a common user cannot achieve a degree of understanding and correctly setting in a short time, and a certain difficulty is brought to initialization of the database, in one or more embodiments of the present disclosure, before determining whether the database system triggers an optimization thread based on the basic configuration information and the processing event response information, the method further includes the following steps:
firstly, storing basic configuration information in a designated configuration file according to a preset first format; wherein, it should be noted that the basic configuration information includes: CPU information, memory information, hard disk information and network card information. And the preset first format is used for defining separation parameters of each basic configuration information storage process, namely, a format for determining a separator and a separation width. The basic configuration information is stored in a designated configuration file such as configuration file 1 in fig. 2 after being separated according to a fixed format as shown in fig. 2, and then the program processing is waited, so that the basic configuration information can be initialized later. And setting corresponding scripts according to the initialization suggestions of the parameters, so as to determine initialization parameter values of the configuration parameters in the database system according to the corresponding scripts. It should be noted that the initialization suggestion is set corresponding to the basic configuration information. For example: setting database configuration parameters according to the following initialization suggestions through scripts:
shared_buffers are shared buffers for storing data read from disk, and their initialization is recommended as: is set to one quarter or one eighth of the total memory but less than 8GB.
The effective_cache_size is an estimated value of the system memory, and is used for telling the PostgreSQL the capability of caching system data, and the initialization proposal is as follows: it is set to half of the physical memory.
work _ mem is used to perform the amount of memory for ordering and hashing operations in each query. Its initialization proposal is to set it to 5% of the total memory.
The main_work_mem is the amount of memory used to perform the VACUUM, CREATE INDEX and ALTER TABLE operations, and its initialization proposal is to set it to 1% of the total amount of memory.
checkpoint_completion_target is a target ratio for setting the checkpoint completion time, whose initialization suggestion is to set it to 0.9.
max_ WAL _size and min_ WAL _size are maximum and minimum sizes for setting the WAL log to ensure that checkpointing occurs when the log is full. Its initialization proposal is to set it to half of the maximum available memory.
max_worker_processes and max_parallel_works_per_gather are the maximum number of processes for setting parallel queries and the maximum number of parallel work processes per parallel query. Its initialization proposal is to set it to half the number of hardware threads.
The configuration parameters of the database system are configured based on the numerical value of the configuration data of each current component in the basic configuration information of the server hard disk, and the database system can be seen to be configured and operated depending on the performance of the server hard disk. And initializing the database system according to the initialization parameter value determined in the process, and restarting the initialized database to finish the initialization of the database.
S103: if the optimization thread is triggered, collecting performance data of the database system; wherein the performance data comprises: performance metrics and workload parameters.
If it is determined that the database system needs to be updated from the aspect of the current server basic configuration information and the personnel requirement based on the above step S102, an optimization system of the database system is triggered. In the embodiment of the present description, if the optimization thread is triggered, performance data of the database system is collected. The following description is needed: the performance data includes performance metrics and workload parameters, the performance data including: CPU utilization, memory usage, disk I/O, etc., workload parameters such as query type, query time consumption, number of concurrent connections, etc.
Specifically, in one or more embodiments of the present description, collecting performance data of a database system specifically includes the steps of:
firstly, according to a preset command of a database system, collecting performance data in a unit time period of the database system, and then, in order to be capable of being executed by a program, storing the performance data into a specified performance file according to a preset second format. Wherein, it should be noted that the second format is preset for defining the separator and the separation width of each performance data storage process. Further, in one or more embodiments of the present disclosure, based on a preset command of the database system, collecting performance data of the database system in a unit time period specifically includes the following steps:
firstly, according to a built-in monitoring tool or a third-party tool of the database system, collecting performance indexes of the database system. And then acquiring the log file in the database system, thereby extracting the workload information of the database system in the log file. Wherein the workload information includes: query type, query time consumption, number of concurrent connections. And then extracting the performance index and the workload information in a preset unit time period to serve as performance data in order to unify the acquisition time of the performance index and the workload information.
Further, in one or more embodiments of the present disclosure, according to a built-in monitoring tool or a third party tool of the database system, the performance index of the database system is collected, which specifically includes the following procedures:
according to a monitoring tool or a third party tool built in PostgreSQL, a first collection command such as a top command or a htop command such as pgAdmin, PMM, prometheus is used for obtaining the CPU utilization rate of the database system, and based on a preset unitization command such as awk command, the average value of the CPU utilization rate in a unit time period is obtained; and acquiring the memory use condition according to a second collection command, such as a free-m command, of the built-in monitoring tool or the third party tool, and calculating the percentage of the used memory to the total memory so as to determine the memory use rate of the database system in the acquisition unit time period. And meanwhile, according to a third collection command, such as a command iostat, of the built-in monitoring tool or the third party tool, reading and writing information of the database system is obtained, so that the throughput of the disk network in the unit time period of the database system is determined according to the obtained reading and writing information. Disk network throughput per unit time period may be obtained by commanding an iftop or nload. The read-write information includes: parameters such as read-write rate per second, read-write delay, etc. And then taking the average value of CPU utilization rate in the unit time period, the memory utilization rate of the database system in the unit time period and the disk network throughput in the unit time period as performance indexes of the database system.
S104: and preprocessing the performance data to obtain training data, and training an initial supervised learning model based on the training data to obtain a parameter prediction model.
In order to automatically predict and modify the configuration parameters, in the embodiment of the present disclosure, the performance data needs to be preprocessed, so as to obtain training data, so as to train the initial supervised learning model according to the training data, and obtain the parameter prediction model.
Specifically, in one or more embodiments of the present disclosure, preprocessing performance data to obtain training data to train an initial supervised learning model based on the training data to obtain a parameter prediction model, specifically including the steps of:
firstly, data cleaning is carried out on the performance data to obtain filtered performance data, so that key performance data of the filtered performance data are extracted. And then, carrying out data scaling on the key performance data according to the input size of the initial supervised learning model, thereby obtaining training data. And then dividing the training data into a training set and a testing set so as to train the initial supervised learning model according to the training set and obtain a trained supervised learning model. And if the trained supervised learning model is evaluated to meet the requirements according to the test set, taking the trained supervised learning model as a parameter prediction model meeting the requirements.
Further, in order to save computational analysis costs and obtain key performance data parameters, in one or more embodiments of the present disclosure, the key performance data of the filtered performance data is extracted, which specifically includes the following steps:
first, a plurality of associated computing nodes connected in series with the computing node where the database system is located are obtained. And then acquiring historical configuration parameter adjustment records of the database system and the database system of the associated computing node. And clustering the performance data in the historical configuration parameter adjustment record according to the data characteristics thereof to obtain clustering sets of each performance data, and determining the first key performance data based on the quantity of the performance data in each clustering set. Wherein the data features are used to characterize the influencing object of the performance data, it is understood that the greater the number of identical performance data in the cluster set, the higher the criticality of the performance data. Since the first performance data is empirically obtained based on historical data, after the first performance data is determined, in order to be able to determine second performance data reflecting the current performance further based on the change in the current performance parameter. In the embodiment of the specification, historical performance data of a plurality of historical unit time periods are obtained, so that the historical performance data and the performance data are compared in pairs based on time sequence, and a differential data set is obtained; here, it is to be noted that the history unit time period corresponds to the unit time period. And then obtaining mutation data of each performance data in each differential data set, and if the numerical value of the mutation data is determined to be larger than the preset change frequency of the performance data, taking the performance data corresponding to the mutation data as second key performance data. And determining an abnormal value of the change in the unit time of the performance data as second key performance data according to the change in the unit time of the performance data and the change in the unit time of the history, and then determining the key performance data of the filtered performance data according to the union of the first key performance data and the second key performance data. The change condition of the current performance parameters is fully combined, so that the determined key performance parameters are fused with historical experience and can reflect the current characteristics.
S105: and inputting each configuration parameter of the database system and a preset value range thereof into the parameter prediction model so as to output initial configuration parameters.
After determining the parameter prediction model based on the step S104, each configuration parameter of the database system and its preset value range can be input into the parameter prediction model to output the initial configuration parameter predicted by the parameter prediction model, and in order to further optimize the initial configuration parameter determined by the model, it is further required to perform iterative optimization according to a subsequent preset genetic optimization algorithm, so as to improve the accuracy of the optimization.
S106: and iterating the initial configuration parameters based on a preset genetic optimization algorithm to obtain the optimal configuration parameters corresponding to the database system so as to perform optimization recommendation.
After obtaining the initial configuration parameters according to the steps, the embodiment of the specification iterates the initial configuration parameters according to a preset genetic optimization algorithm, so as to obtain the optimal configuration parameters corresponding to the database system and further optimize and recommend the optimal configuration parameters.
Specifically, in one or more embodiments of the present disclosure, iterating the initial configuration parameters based on a preset genetic optimization algorithm to obtain optimal configuration parameters corresponding to the database system for optimization recommendation, including:
Randomly generating a preset number of configuration parameter combinations in a preset value range of each configuration parameter to obtain an initialization population; wherein, it should be noted that each configuration parameter is combined into an individual in the initialized population. And then, obtaining performance indexes corresponding to each configuration parameter combination, and taking the corresponding performance indexes as fitness functions of the initialized population. And then determining parent individuals of the initialized population according to the roulette selection mode, so as to carry out cross operation on the parent individuals based on single-point cross and obtain child individuals of the initialized population. The basic idea is that: the probability of each individual being selected is proportional to its fitness level and is not specifically described herein for the present selection. And carrying out mutation operation on the child individuals according to the random mutation mode to obtain variant child individuals of the initial population, so as to determine fitness values corresponding to the variant child individuals based on the fitness function. If the fitness value is smaller than the preset fitness value, a current population is formed according to the child individuals, the variant child individuals and the preset number of parent individuals, and accordingly iterative operation is conducted on the current population to obtain the configuration parameter combination corresponding to the optimal individual. And combining the configuration parameters corresponding to the optimal individuals to serve as the optimal configuration parameters corresponding to the database system, so that the configuration parameters of the database system are updated and replaced according to the optimal configuration parameters.
Specifically, in a certain application scenario of the present specification, parameters of the PostgreSQL database are automatically optimized with a reliable model. The method comprises the following steps: firstly, defining a value range of parameters according to a parameter configuration file of PostgreSQL, and completing definition of a parameter space. The optimal parameter configuration is then searched using a genetic algorithm. Firstly, determining an initial parameter configuration through model prediction; then, the genetic algorithm is used to iterate continuously, generating new parameter configurations, and evaluating their performance until the best parameter configuration is found. The basic flow using genetic algorithm is as follows:
s1: initializing a population: a number of parameter combinations are randomly generated as a population.
S2: calculating the fitness: each parameter combination is used as an individual, and the fitness function is calculated. The fitness function is typically a performance indicator such as query response time, CPU utilization, etc.
S3: selection operation: through the selection operation, a number of individuals are selected from the population, typically using roulette selection or tournament selection as the parent of the next generation.
S4: crossover operation: the parent is interleaved to generate a number of children, typically using single-point or multi-point interleaving.
S5: mutation operation: the progeny are subjected to a mutation operation to generate a number of new individuals, typically using random or non-uniform mutation.
S6: calculating the fitness: and calculating the fitness function of the new individual.
S7: screening operation: through the selection operation, a certain number of individuals are selected from the parent and the offspring to form a new population.
S8: termination condition: and when the preset iteration times or the fitness meets the preset requirements, stopping the algorithm and outputting an optimal solution.
Further, in the case of multiple computing nodes, in order to enable the current computing node to be optimized, the method can be synchronized to other computing nodes to provide auxiliary references for optimizing the other computing nodes, so that the problem of computing resource waste caused by repeated analysis is avoided. In one or more embodiments of the present disclosure, after iterating the initial configuration parameters according to a preset genetic optimization algorithm to obtain the optimal configuration parameters corresponding to the database system for optimization recommendation, the method further includes the following steps:
the method comprises the steps of obtaining first model data of each hardware configuration in basic configuration information of an associated computing node, and obtaining second model data of each hardware configuration in the basic configuration information of the computing node where a database system is located. If it can be determined that the first model data is consistent with the second model data, synchronizing the optimal configuration parameters of the computing node where the database system is located and the performance data of the database system corresponding to the optimal configuration parameters to the associated computing node. Under the condition that the performance data of the associated computing node is matched with the performance data of the database system corresponding to the optimal configuration parameters, the database system of the associated computing node can be directly subjected to parameter optimization according to the optimal configuration parameters, so that the optimization efficiency of the associated computing node is improved, and the analysis cost is reduced.
As shown in fig. 3, in one or more embodiments of the present disclosure, there is provided an apparatus for adaptive optimization of database parameters, the apparatus including:
at least one processor 301; the method comprises the steps of,
a memory 302 communicatively coupled to the at least one processor 301; wherein,,
the memory 302 stores instructions executable by the at least one processor 301, the instructions being executable by the at least one processor 301 to enable the at least one processor 301 to:
collecting basic configuration information of server hardware based on an information acquisition command, and acquiring processing event response information of a corresponding database system of the server within a preset time period;
judging whether the database system triggers an optimization thread or not based on the basic configuration information and the processing event response information;
if the optimization thread is triggered, collecting performance data of the database system; wherein the performance data comprises: performance index and workload parameters;
preprocessing the performance data to obtain training data, and training an initial supervised learning model based on the training data to obtain a parameter prediction model;
inputting each configuration parameter of the database system and a preset value range thereof into the parameter prediction model to output initial configuration parameters;
And iterating the initial configuration parameters based on a preset genetic optimization algorithm to obtain the optimal configuration parameters corresponding to the database system so as to perform optimization recommendation.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for apparatus, devices, non-volatile computer storage medium embodiments, the description is relatively simple, as it is substantially similar to method embodiments, with reference to the section of the method embodiments being relevant.
The foregoing describes specific embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
The foregoing is merely one or more embodiments of the present description and is not intended to limit the present description. Various modifications and alterations to one or more embodiments of this description will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, or the like, which is within the spirit and principles of one or more embodiments of the present description, is intended to be included within the scope of the claims of the present description.
Claims (11)
1. A method for adaptive optimization of database parameters, the method comprising:
collecting basic configuration information of server hardware based on an information acquisition command, and acquiring processing event response information of a corresponding database system of the server within a preset time period;
judging whether the database system triggers an optimization thread or not based on the basic configuration information and the processing event response information;
if the optimization thread is triggered, collecting performance data of the database system; wherein the performance data comprises: performance index and workload parameters;
preprocessing the performance data to obtain training data, and training an initial supervised learning model based on the training data to obtain a parameter prediction model;
Inputting each configuration parameter of the database system and a preset value range thereof into the parameter prediction model to output initial configuration parameters;
and iterating the initial configuration parameters based on a preset genetic optimization algorithm to obtain the optimal configuration parameters corresponding to the database system so as to perform optimization recommendation.
2. The method for adaptively optimizing database parameters according to claim 1, wherein the iterating the initial configuration parameters based on a preset genetic optimization algorithm to obtain optimal configuration parameters corresponding to the database system for optimization recommendation specifically comprises:
randomly generating a preset number of configuration parameter combinations in a preset value range of each configuration parameter to obtain an initialization population; wherein each of the configuration parameters are combined into an individual in the initialized population;
acquiring performance indexes corresponding to the configuration parameter combinations, and taking the corresponding performance indexes as fitness functions of the initialized population;
determining parent individuals of the initialized population based on a roulette selection mode, so as to perform cross operation on the parent individuals based on single-point cross and obtain child individuals of the initialized population;
Performing mutation operation on the child individuals according to a random mutation mode to obtain variant child individuals of the initialized population, so as to determine fitness values corresponding to the variant child individuals based on the fitness function;
if the fitness value is smaller than a preset fitness value, forming a current population based on the child individuals, the variant child individuals and a preset number of parent individuals, so as to obtain a configuration parameter combination corresponding to the optimal individual by performing iterative operation on the current population;
and combining the configuration parameters corresponding to the optimal individuals to serve as optimal configuration parameters corresponding to the database system, so that the configuration parameters of the database system are updated and replaced based on the optimal configuration parameters.
3. The method of claim 1, wherein prior to determining whether the database system triggers an optimization thread based on the base configuration information and the processing event response information, the method further comprises:
storing the basic configuration information in a designated configuration file based on a preset first format; wherein the basic configuration information includes: CPU information, memory information, hard disk information and network card information; the preset first format is used for defining separation parameters of each basic configuration information storage process; the separation parameters include: a separator and a separation width;
Setting corresponding scripts based on initialization suggestions of preset parameters so as to determine initialization parameter values of configuration parameters in the database system based on the scripts; wherein the initialization suggestion is set corresponding to the basic configuration information;
and initializing the database system based on the initialization parameter value, and restarting the initialized database.
4. A method of adaptive optimization of database parameters according to claim 1, wherein collecting performance data of the database system comprises:
collecting performance data in a unit time period of the database system based on a preset command of the database system, and storing the performance data in a designated performance file based on a preset second format; wherein the preset second format is used for defining a separator and a separation width of each performance data storage process.
5. The method for adaptively optimizing database parameters according to claim 4, wherein said collecting performance data within a unit time period of said database system based on a preset command of said database system specifically comprises:
collecting performance indexes of the database system according to a built-in monitoring tool or a third-party tool of the database system;
Acquiring a log file of the database system to extract the workload information of the database system in the log file; wherein the workload information includes: query type, time consuming query, number of concurrent connections;
and extracting the performance index and the workload information within a preset unit time period as performance data.
6. The method for adaptively optimizing database parameters according to claim 5, wherein collecting performance indexes of the database system according to a built-in monitoring tool or a third party tool of the database system specifically comprises:
based on a first collection command of the built-in monitoring tool or the third party tool, obtaining the CPU utilization rate of the database system, and based on a preset unitization command, obtaining an average value of the CPU utilization rate in a unit time period; and
acquiring the memory utilization rate of the database system in a unit time period according to the second collection command of the built-in monitoring tool or the third party tool; and
acquiring read-write information of the database system according to a third collection command of the built-in monitoring tool or the third party tool so as to acquire disk network throughput in a unit time period of the database system based on the read-write information; wherein the read-write information includes: a read-write rate per second, a read-write delay;
And taking the average value of the CPU utilization rate in the unit time period, the memory utilization rate of the database system in the unit time period and the disk network throughput in the unit time period as performance indexes of the database system.
7. The method for adaptive optimization of database parameters according to claim 1, wherein preprocessing the performance data to obtain training data, so as to train an initial supervised learning model based on the training data, to obtain a parameter prediction model, specifically comprising:
performing data cleaning on the performance data to obtain filtered performance data so as to extract key performance data of the filtered performance data;
based on the input size of the initial supervised learning model, performing data scaling on the key performance data to obtain training data;
dividing the training data into a training set and a testing set, so as to train the initial supervised learning model based on the training set and obtain a trained supervised learning model;
and if the trained supervised learning model meets the requirements based on the test set, taking the trained supervised learning model as a parameter prediction model meeting the requirements.
8. The method for adaptively optimizing database parameters according to claim 7, wherein extracting key performance data of said filtered performance data comprises:
acquiring a plurality of associated computing nodes connected in series with the computing node where the database system is located;
acquiring historical configuration parameter adjustment records of the database system and the database system of the associated computing node;
clustering the performance data in the historical configuration parameter adjustment record based on data characteristics to obtain clustering sets of the performance data, and determining first key performance data based on the number of the performance data in each clustering set; wherein the data features are used to characterize the influencing object of the performance data;
acquiring historical performance data of a plurality of historical unit time periods, and comparing the historical performance data with the performance data in pairs based on time sequence to obtain a differential data set; wherein the historical unit time period corresponds to the unit time period;
obtaining mutation data of each performance data in the differential data set, and taking the performance data corresponding to the mutation data as second key performance data if the numerical value of the mutation data is larger than the preset variation frequency of the performance data;
And determining the key performance data of the filtered performance data according to the union set of the first key performance data and the second key performance data.
9. The method for adaptively optimizing database parameters according to claim 8, wherein determining whether the database system triggers an optimization thread based on the basic configuration information and the processing event response information comprises:
acquiring current component configuration data of the server hardware in the basic configuration information, and triggering the optimization thread if the current component configuration data does not exist in a preset component configuration data range;
if the current component configuration data exists in the preset component configuration data range, determining that the computing node depends on an application program of the database system;
acquiring the work order processing information and the application evaluation information uploaded in the preset time period of each application program, and taking the work order processing information and the application evaluation information as processing event response information;
splitting the work order processing information based on a preset word splitting length to obtain a plurality of word splitting words, and filtering connective word words in the work order processing information based on semantic attributes of the word splitting words to obtain keywords to be identified of the work order processing information;
Acquiring preset description information corresponding to each configuration parameter, determining the number of the configuration parameters corresponding to the work order processing information based on the matching relation between the keywords to be identified and the preset description information, and determining a first optimized weight value based on the number of the configuration parameters corresponding to the work order processing information;
based on the evaluation value of the application evaluation information, extracting corresponding negative evaluation information to preprocess the negative evaluation information and obtain keywords to be analyzed associated with the basic configuration information;
determining a second optimized weight value based on the ratio of the negative evaluation information of the keywords to be analyzed to all the negative evaluation information and the number of the keywords to be analyzed;
and determining the weight value of the basic configuration information triggering optimization thread based on the first optimization weight value and the second optimization weight value, wherein the weight value is larger than a preset weight value and triggers the optimization thread.
10. The method for adaptively optimizing parameters of a database according to claim 8, wherein after said iterating said initial configuration parameters based on a preset genetic optimization algorithm to obtain an optimal configuration parameter corresponding to said database system for optimization recommendation, said method further comprises:
Acquiring first model data of each hardware configuration in basic configuration information of the associated computing node, and acquiring second model data of each hardware configuration in the basic configuration information of the computing node where the database system is located;
if the first model data is consistent with the second model data, synchronizing the optimal configuration parameters and performance data of the database system corresponding to the optimal configuration parameters to the associated computing node;
and if the performance data of the associated computing node is matched with the performance data of the database system corresponding to the optimal configuration parameter, optimizing the database system of the associated computing node based on the optimal configuration parameter.
11. An apparatus for adaptive optimization of database parameters, the apparatus comprising:
at least one processor; the method comprises the steps of,
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores instructions executable by the at least one processor to enable the at least one processor to:
collecting basic configuration information of server hardware based on an information acquisition command, and acquiring processing event response information of a corresponding database system of the server within a preset time period;
Judging whether the database system triggers an optimization thread or not based on the basic configuration information and the processing event response information;
if the optimization thread is triggered, collecting performance data of the database system; wherein the performance data comprises: performance index and workload parameters;
preprocessing the performance data to obtain training data, and training an initial supervised learning model based on the training data to obtain a parameter prediction model;
inputting each configuration parameter of the database system and a preset value range thereof into the parameter prediction model to output initial configuration parameters;
and iterating the initial configuration parameters based on a preset genetic optimization algorithm to obtain the optimal configuration parameters corresponding to the database system so as to perform optimization recommendation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310665655.XA CN116401236B (en) | 2023-06-07 | 2023-06-07 | Method and equipment for adaptively optimizing database parameters |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310665655.XA CN116401236B (en) | 2023-06-07 | 2023-06-07 | Method and equipment for adaptively optimizing database parameters |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116401236A true CN116401236A (en) | 2023-07-07 |
| CN116401236B CN116401236B (en) | 2023-08-18 |
Family
ID=87009087
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310665655.XA Active CN116401236B (en) | 2023-06-07 | 2023-06-07 | Method and equipment for adaptively optimizing database parameters |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116401236B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117539728A (en) * | 2024-01-10 | 2024-02-09 | 浙江睿数云联科技有限公司 | Distributed database monitoring realization method, system, device and medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1543092A (en) * | 2003-11-07 | 2004-11-03 | 清华大学 | SCSI instruction optimization method based on FC-SAN storage device |
| CN111930484A (en) * | 2020-07-24 | 2020-11-13 | 国家电网有限公司 | Method and system for optimizing performance of thread pool of power grid information communication server |
| US20210216351A1 (en) * | 2020-01-15 | 2021-07-15 | Purdue Research Foundation | System and methods for heterogeneous configuration optimization for distributed servers in the cloud |
| CN113626463A (en) * | 2021-07-31 | 2021-11-09 | 西南电子技术研究所（中国电子科技集团公司第十研究所） | Web performance optimization method under high concurrent access |
| CN115204064A (en) * | 2022-09-19 | 2022-10-18 | 中国石油大学(华东) | Gas injection huff-puff parameter optimization method and system in shale oil exploitation |
| CN116204503A (en) * | 2021-12-01 | 2023-06-02 | 中兴通讯股份有限公司 | Database parameter tuning method, network device and computer readable storage medium |
-
2023
- 2023-06-07 CN CN202310665655.XA patent/CN116401236B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1543092A (en) * | 2003-11-07 | 2004-11-03 | 清华大学 | SCSI instruction optimization method based on FC-SAN storage device |
| US20210216351A1 (en) * | 2020-01-15 | 2021-07-15 | Purdue Research Foundation | System and methods for heterogeneous configuration optimization for distributed servers in the cloud |
| CN111930484A (en) * | 2020-07-24 | 2020-11-13 | 国家电网有限公司 | Method and system for optimizing performance of thread pool of power grid information communication server |
| CN113626463A (en) * | 2021-07-31 | 2021-11-09 | 西南电子技术研究所（中国电子科技集团公司第十研究所） | Web performance optimization method under high concurrent access |
| CN116204503A (en) * | 2021-12-01 | 2023-06-02 | 中兴通讯股份有限公司 | Database parameter tuning method, network device and computer readable storage medium |
| CN115204064A (en) * | 2022-09-19 | 2022-10-18 | 中国石油大学(华东) | Gas injection huff-puff parameter optimization method and system in shale oil exploitation |
Non-Patent Citations (2)
| Title |
|---|
| YOUCEF DJENOURI等: "Exploiting GPU parallelism in improving bees swarm optimization for mining big transactional databases", 《INFORMATION SCIENCES》, vol. 496, pages 326 - 342, XP085705970, DOI: 10.1016/j.ins.2018.06.060 * |
| 李运莉;: "基于负荷预测的数据库连接池设计与实现", 计算机与数字工程, no. 09, pages 144 - 146 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117539728A (en) * | 2024-01-10 | 2024-02-09 | 浙江睿数云联科技有限公司 | Distributed database monitoring realization method, system, device and medium |
| CN117539728B (en) * | 2024-01-10 | 2024-04-12 | 浙江睿数云联科技有限公司 | Distributed database monitoring realization method, system, device and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116401236B (en) | 2023-08-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6842440B2 (en) | Performance analysis method and management computer | |
| Tan et al. | ibtune: Individualized buffer tuning for large-scale cloud databases | |
| CN110956272B (en) | Method and system for realizing data processing | |
| US6622221B1 (en) | Workload analyzer and optimizer integration | |
| CN110704542A (en) | Data dynamic partitioning system based on node load | |
| CN108052528A (en) | A kind of storage device sequential classification method for early warning | |
| CN116401236B (en) | Method and equipment for adaptively optimizing database parameters | |
| CN110389820B (en) | Private cloud task scheduling method for resource prediction based on v-TGRU model | |
| CN109492826B (en) | Information system running state risk prediction method based on machine learning | |
| US10410137B2 (en) | Method and system for analyzing accesses to a data storage type and recommending a change of storage type | |
| US20220179884A1 (en) | Label Determining Method, Apparatus, and System | |
| WO2021185206A1 (en) | Resource allocation method and apparatus for cluster task, and computer apparatus and storage medium | |
| US10963802B1 (en) | Distributed decision variable tuning system for machine learning | |
| US20220067008A1 (en) | Method and apparatus for determining configuration knob of database | |
| CN115756901A (en) | Business decision processing method and rule engine system thereof | |
| WO2023016537A1 (en) | Database management system, data processing method, and device | |
| CN112052082B (en) | Task attribute optimization method, device, server and storage medium | |
| CN111651220A (en) | Spark parameter automatic optimization method and system based on deep reinforcement learning | |
| CN113673822A (en) | Elastic scheduling method and system | |
| CN112328254A (en) | Performance optimization method and device of data synchronization software | |
| Nikitopoulou et al. | Performance analysis and auto-tuning for spark in-memory analytics | |
| CN116185797A (en) | Method, device and storage medium for predicting server resource saturation | |
| CN115860141A (en) | Automatic machine learning interactive black box visual modeling method and system | |
| CN115543671A (en) | Data analysis method, device, equipment, storage medium and program product | |
| CN112395167A (en) | Operation fault prediction method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |