CN113590576A

CN113590576A - Database parameter adjusting method and device, storage medium and electronic equipment

Info

Publication number: CN113590576A
Application number: CN202110164625.1A
Authority: CN
Inventors: 张光钰; 蔡宝清; 周可; 李春花; 程彬; 邢家树
Original assignee: Tencent Technology Shenzhen Co Ltd; Huazhong University of Science and Technology
Current assignee: Tencent Technology Shenzhen Co Ltd; Huazhong University of Science and Technology
Priority date: 2021-02-05
Filing date: 2021-02-05
Publication date: 2021-11-02

Abstract

The embodiment of the application discloses a database parameter adjusting method, a database parameter adjusting device, a storage medium and electronic equipment, and relates to the fields of artificial intelligence, cloud technology, big data and the like. The method comprises the following steps: acquiring a parameter adjustment request for a database to be optimized; acquiring initial configuration information corresponding to each database parameter of the database to be optimized according to the parameter adjustment request; determining at least two initial configuration combinations corresponding to each database parameter according to the initial configuration information corresponding to each database parameter; and respectively performing performance test on the database to be optimized based on at least two initial configuration combinations, determining a target configuration combination corresponding to each database parameter based on a performance test result corresponding to each initial configuration combination, and updating each database parameter of the database to be optimized based on the target configuration combination. By adopting the technical scheme, the adjustment efficiency of the database parameters can be effectively improved.

Description

Database parameter adjusting method and device, storage medium and electronic equipment

Technical Field

The application relates to the fields of artificial intelligence, cloud technology, big data and the like, in particular to a database parameter adjusting method and device, a storage medium and electronic equipment.

Background

With the increasing speed of data volume in the information age increasing continuously and services becoming complicated, the requirements of high throughput and low delay are brought to the database, and good configuration parameters guarantee the performance of the database.

It may be desirable for a user to find an available database parameter as quickly as possible in a short period of time while using the database. The database parameter optimization method can be used for performing optimization in a manual optimization mode, however, along with updating iteration of the database, the database parameters are more and more complex, the types of the database are more diversified, and a database administrator needs to perform parameter optimization through understanding and experience of the database, so that high requirements are provided for the database administrator, and the learning cost of the database administrator is continuously increased. However, for most database parameters, when a new load is encountered, the database parameters need to undergo long training, and the requirements of a user side are urgent, so that a better solution of the database parameters cannot be given in a short time.

Therefore, how to improve the efficiency of tuning database parameters becomes an urgent problem to be solved.

Disclosure of Invention

The embodiment of the application provides a database parameter adjusting method, a database parameter adjusting device, a storage medium and electronic equipment, and the adjusting efficiency of database parameters can be effectively improved.

In one aspect, an embodiment of the present application provides a database parameter adjustment method, where the method includes:

acquiring a parameter adjustment request for a database to be optimized;

acquiring initial configuration information corresponding to each database parameter of the database to be optimized according to the parameter adjustment request;

determining at least two initial configuration combinations corresponding to the database parameters according to the initial configuration information corresponding to the database parameters;

and respectively performing performance test on the database to be optimized based on the at least two initial configuration combinations, determining a target configuration combination corresponding to each database parameter based on a performance test result corresponding to each initial configuration combination, and updating each database parameter of the database to be optimized based on the target configuration combination.

In one aspect, an embodiment of the present application provides a database parameter adjusting apparatus, where the apparatus includes:

the parameter adjustment request acquisition module is used for acquiring a parameter adjustment request aiming at the database to be optimized;

an initial configuration information obtaining module, configured to obtain initial configuration information corresponding to each database parameter of the database to be optimized according to the parameter adjustment request;

a configuration combination determining module, configured to determine at least two initial configuration combinations corresponding to the database parameters according to the initial configuration information corresponding to the database parameters;

and the target configuration parameter determining module is used for respectively performing performance test on the database to be optimized based on the at least two initial configuration combinations, determining a target configuration combination corresponding to each database parameter based on a performance test result corresponding to each initial configuration combination, and updating each database parameter of the database to be optimized based on the target configuration combination.

In an optional embodiment, the target configuration parameter determining module is configured to:

repeatedly executing the following operations based on the at least two initial configuration combinations until a preset end condition is met, and determining the target configuration combination based on each obtained initial configuration combination:

performing performance test on each initial configuration combination to obtain a performance test result corresponding to each initial configuration combination;

determining initial configuration combinations from the initial configuration combinations based on performance test results corresponding to the initial configuration combinations;

and determining at least two new configuration combinations corresponding to the database parameters according to the initial configuration information corresponding to the database parameters, and taking the at least two new configuration combinations as the at least two initial configuration combinations.

filtering each initial configuration combination based on a performance test result corresponding to each initial configuration combination and a preset performance constraint condition to obtain each reserved initial configuration combination;

and determining the initial configuration combination from the reserved initial configuration combinations based on the performance test result corresponding to the reserved initial configuration combinations.

acquiring optimization constraint conditions corresponding to the database to be optimized;

determining each initial configuration combination which meets the optimization constraint condition in each initial configuration combination, and determining an initial selection configuration combination according to the performance test result of each initial configuration combination which meets the optimization constraint condition;

or determining candidate configuration combinations according to performance test results corresponding to the initial configuration combinations, and determining the combination meeting the optimization constraint condition in the candidate configuration combinations as the initial selection configuration combination.

In an optional embodiment, the preset ending condition includes at least one of the following:

the operation time length of the operation reaches a first set time length;

the initial selection configuration combinations corresponding to at least two times of operations are consistent;

the operation time length of the operation does not exceed a second set time length, and the initial selection configuration combinations corresponding to the operation for at least two times are consistent;

and receiving an operation ending instruction.

In an optional embodiment, the target configuration parameter determining module is configured to at least one of:

determining at least one target configuration combination according to the performance test result of each initially selected configuration combination;

respectively taking each initially selected configuration combination as a target configuration combination;

and acquiring optimization constraint conditions corresponding to the database to be optimized, and determining at least one target configuration combination according to the performance test result of each initially selected configuration combination and the optimization constraint conditions.

and respectively applying each initial configuration combination to the database instance of the database to be optimized, and carrying out performance test on the database instance corresponding to each initial configuration combination.

the performing a performance test on the database instance corresponding to each initial configuration combination includes:

acquiring historical service data of the database to be optimized;

and generating a test case based on the historical service data, and performing performance test on the database instance corresponding to each initial configuration combination based on the test case.

and distributing the at least two initial configuration combinations to at least two test devices, and performing performance test on the database to be optimized in parallel through the at least two test devices based on the respective distributed initial configuration combinations.

respectively performing performance test on the database to be optimized based on each initial configuration combination to obtain performance test information corresponding to each initial configuration combination, wherein the performance test information comprises at least one of test throughput or test delay;

acquiring performance benchmark information corresponding to the database to be optimized, wherein the performance benchmark information comprises at least one of benchmark throughput or benchmark delay;

and for each initial configuration combination, determining a performance test result corresponding to the initial configuration combination according to the performance standard information and the performance test information corresponding to the initial configuration combination.

for each initial configuration combination, the determining a performance test result corresponding to the initial configuration combination according to the performance benchmark information and the performance test information corresponding to the initial configuration combination includes:

acquiring a first weight corresponding to throughput and a second weight corresponding to delay;

and determining a performance test result corresponding to the initial configuration combination according to the performance test information, the performance reference information, the first weight and the second weight.

In one aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, where the processor and the memory are connected to each other; the memory is used for storing a computer program; the processor is configured to execute the method provided by any possible implementation manner of the database parameter adjustment method when the computer program is called.

In one aspect, the present application provides a computer-readable storage medium, which stores a computer program, where the computer program is executed by a processor to implement the method provided in any one of the possible implementation manners of the above database parameter adjustment method.

In one aspect, embodiments of the present application provide a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided by any possible implementation manner of the database parameter adjusting method.

The beneficial effect that technical scheme that this application provided brought is:

in this embodiment of the present application, after a parameter adjustment request for a database to be optimized is obtained, initial configuration information of each database parameter of data to be optimized may be obtained based on the request, according to the database parameter adjustment method, the database parameter adjustment device, the storage medium, and the electronic device provided in this embodiment of the present application. Then, at least two possible initial configuration combinations of the database parameters are generated according to the initial configuration information. Then, based on the at least two initial configuration combinations, testing the performance of the data to be optimized to obtain a performance test result corresponding to each initial configuration combination, and then based on the performance test result, determining a target configuration combination corresponding to each database parameter, so that each database parameter of the database to be optimized can be updated based on the target configuration combination. By adopting the method, a feasible target configuration combination can be determined on the premise of ensuring the performance requirement, the problem that in the related technology, long early-stage training is needed, and then parameter optimization is carried out based on a trained model is avoided, so that the efficiency of adjusting each database parameter of the database to be optimized is greatly improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

Fig. 1 is a schematic implementation flow diagram of a database parameter adjustment system in an application scenario according to an embodiment of the present application;

fig. 2 is a schematic flowchart of a database parameter adjustment method according to an embodiment of the present application;

FIG. 3 is a schematic diagram of an overall framework of a database parameter adjustment method according to an embodiment of the present disclosure;

fig. 4 is a schematic structural diagram of a database parameter adjustment apparatus according to an embodiment of the present disclosure;

fig. 5 is a schematic diagram of an operating principle of a management end according to an embodiment of the present application;

FIG. 6 is a schematic diagram illustrating an operation of a sub-machine for performance testing according to an embodiment of the present disclosure;

fig. 7 is a schematic flowchart illustrating a process of updating and eliminating a parameter set by way of health monitoring according to an embodiment of the present application;

fig. 8 is a schematic flowchart of a performance test performed by a real load according to an embodiment of the present application;

FIG. 9 is a schematic flow chart illustrating a method for determining a target configuration combination according to an embodiment of the present application;

fig. 10 is an alternative structural diagram of the distributed system 100 applied to the blockchain system according to the embodiment of the present application;

FIG. 11 is an alternative Block Structure (Block Structure) diagram provided in an embodiment of the present application;

FIG. 12 is a schematic structural diagram of another database parameter adjustment apparatus according to an embodiment of the present application;

fig. 13 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The database parameter adjusting method provided by the embodiment of the application relates to various fields such as artificial intelligence and Cloud technology, for example, databases, Cloud computing, Cloud services and the like in Cloud technology, and related data computing processing fields in the field of big data.

Artificial Intelligence (AI) is a theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human Intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making.

The cloud technology is a hosting technology for unifying series resources such as hardware, software, network and the like in a wide area network or a local area network to realize the calculation, storage, processing and sharing of data. The database parameter adjustment method provided by the embodiment of the application can be realized based on cloud computing (cloud computing) in cloud technology.

Database (Database), which can be regarded as an electronic file cabinet in short, a place for storing electronic files, a user can add, query, update, delete, etc. to data in files. A "database" is a collection of data that is stored together in a manner that can be shared by multiple users, has as little redundancy as possible, and is independent of the application.

A Database Management System (DBMS) is a computer software System designed for managing a Database, and generally has basic functions of storage, interception, security assurance, backup, and the like. The database management system may classify the database according to the database model it supports, such as relational, XML (Extensible Markup Language); or classified according to the type of computer supported, e.g., server cluster, mobile phone; or sorted according to the Query Language used, such as SQL (Structured Query Language), XQuery, or sorted according to performance impulse emphasis, such as max size, maximum operating speed, or other sorting.

Cloud Computing refers to obtaining required resources in an on-demand and easily-extensible manner through a Network, and is a product of development and fusion of traditional computers and Network Technologies, such as Grid Computing (Grid Computing), Distributed Computing (Distributed Computing), Parallel Computing (Parallel Computing), Utility Computing (Utility Computing), Network Storage (Network Storage Technologies), Virtualization (Virtualization), Load balancing (Load Balance), and the like.

An artificial intelligence cloud Service is also generally called AIaaS (AI as a Service). The method is a service mode of an artificial intelligence platform, and specifically, the AIaaS platform splits several types of common artificial intelligence services, and provides independent or packaged services, such as processing resource conversion requests, at a cloud.

Big data (Big data) refers to a data set which cannot be captured, managed and processed by a conventional software tool within a certain time range, and is a massive, high-growth-rate and diversified information asset which can have stronger decision-making power, insight discovery power and flow optimization capability only by a new processing mode. With the advent of the cloud era, big data has attracted more and more attention. The database parameter adjustment method provided by the embodiment is effectively implemented based on the fact that big data requires special technologies, wherein the technologies suitable for big data include massive parallel processing of a database, data mining, a distributed file system, a distributed database, cloud computing and the like.

As an example, fig. 1 shows an implementation flow diagram of a database parameter adjustment system applied in the embodiment of the present application in an application scenario, and it can be understood that the database parameter adjustment method provided in the embodiment of the present application may be applied to, but is not limited to, the application scenario shown in fig. 1.

The database parameter adjustment system in this example may include, but is not limited to, a user terminal 101, a network 102, and a server 103. A user terminal 101, such as a user's smartphone, may communicate with a server 103 over a network 102. The user terminal 101 may include a human interaction screen 1011, a processor 1012 and a memory 1013. A parameter adjustment request may be initiated on the human-machine interaction screen. For example, the user may initiate a parameter adjustment request for the data to be optimized through the user terminal 101, and send the parameter adjustment request to the server through the network. The server 103 may include a database 1031 and a processing engine 1032, where the processor 1032 is configured to perform corresponding processing after receiving the parameter adjustment request, so as to obtain a target configuration combination corresponding to each database parameter of the data to be optimized. The memory 1031 is used for storing target configuration combinations and the like.

As shown in FIG. 1, the specific interactive process of the database parameter adjustment system in the present application may include steps S1-S4:

in step S1, the user initiates a parameter adjustment request for the database to be optimized through the user terminal 101, and sends the parameter adjustment request to the server 103 through the network 102.

In step S2, the server 103 receives the parameter adjustment request, and the processing engine 1032 obtains initial configuration information corresponding to each database parameter of the database to be optimized according to the parameter adjustment request.

In step S3, the processing engine 1032 in the server 103 determines at least two initial configuration combinations corresponding to the database parameters according to the initial configuration information corresponding to the database parameters.

In step S4, the processing engine 1032 in the server 103 performs performance tests on the databases to be optimized based on the at least two initial configuration combinations, determines a target configuration combination corresponding to each database parameter based on a performance test result corresponding to each initial configuration combination, and sends the target configuration combination to the user terminal.

Step S5, the user terminal receives the target configuration combination, and updates each database parameter of the to-be-optimized database based on the target configuration combination.

The database 1031 in the server 103 is used to store various data generated in steps S2-S4, such as initial configuration information, performance test results corresponding to the initial configuration combinations, target configuration combinations, and the like.

The database parameter adjusting method can be applied to various scenes needing to be subjected to database parameter adjustment, different parameter adjusting requests can be initiated according to business requirements in practical application scenes, and the business requirements can be determined according to the practical application scenes, and are not limited herein. For example, the service may be a shopping application, an instant messaging application, an audio-video application, a search application, and the like, which is not limited herein.

The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server or a server cluster providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a Network service, cloud communication, a middleware service, a domain name service, a security service, a CDN (Content Delivery Network), a big data and artificial intelligence platform, and the like. Such networks may include, but are not limited to: a wired network, a wireless network, wherein the wired network comprises: a local area network, a metropolitan area network, and a wide area network, the wireless network comprising: bluetooth, Wi-Fi, and other networks that enable wireless communication. The user terminal may be a smart phone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a notebook computer, a digital broadcast receiver, an MID (Mobile Internet Devices), a PDA (personal digital assistant), a desktop computer, a vehicle-mounted terminal (e.g., a vehicle-mounted navigation terminal), a smart speaker, a smart watch, etc., and the user terminal and the server may be directly or indirectly connected through wired or wireless communication, but are not limited thereto. The determination may also be based on the requirements of the actual application scenario, and is not limited herein.

Referring to fig. 2, fig. 2 is a schematic flowchart of a database parameter adjustment method provided in an embodiment of the present application, where the method may be executed by any electronic device, such as a server, a user terminal, or an interaction between the user terminal and the server, and optionally may be executed by the server, as shown in fig. 2, the database parameter adjustment method provided in the embodiment of the present application includes the following steps:

step S201, acquiring a parameter adjustment request for a database to be optimized;

step S202, acquiring initial configuration information corresponding to each database parameter of the database to be optimized according to the parameter adjustment request;

step S203, determining at least two initial configuration combinations corresponding to each database parameter according to the initial configuration information corresponding to each database parameter;

step S204, respectively performing performance tests on the database to be optimized based on the at least two initial configuration combinations, determining a target configuration combination corresponding to each database parameter based on a performance test result corresponding to each initial configuration combination, and updating each database parameter of the database to be optimized based on the target configuration combination.

Optionally, the database to be optimized may be determined according to the actual service, for example, the database to be optimized may be a database used by a shopping application, a database used by an instant messaging application, a database used by an audio/video application, a database used by a search application, and the like, which is not limited herein. The DataBase type of the DataBase to be optimized may be a Cloud DataBase (CDB) or a DataBase (DB), and is not limited herein.

The parameter adjustment request may be a HyperText Transfer Protocol (HTTP) type request initiated by a user, or may also be a Remote Procedure Call (RPC) type request, and it is understood that the specific request type of the parameter adjustment request is not limited in this embodiment of the present application. The parameter adjustment request may include a desired performance index (i.e., the performance constraint condition described above), and in practical applications, the hardware performance of the requesting device is limited, and the desired performance index may not necessarily be achieved.

After the parameter adjustment request is obtained, the initial configuration information corresponding to each database parameter of the data to be optimized can be determined according to the parameter adjustment request.

Each database parameter of the database to be optimized may be a parameter that may affect the performance of the database to be optimized, and each database parameter may be determined by a manual screening method, for example, each database parameter may be determined by combining the experience of database management personnel. It is understood that the number of database parameters for the data to be optimized is not limited.

The initial configuration information of each database parameter may be understood as an adjustment interval (also referred to as a value interval) of each database parameter.

The database parameters can be divided into various types, such as integer type, character string type, enumeration type, etc. The initial configuration information corresponding to different types of database parameters is different.

For integer type database parameters, the adjustment interval of any one database parameter is composed of { parameter type, minimum value, maximum value, default value }. The default value is a default parameter value (i.e. a current configuration value) of the database to be optimized, and the minimum value and the maximum value can be a fuzzy interval defined by a database manager according to the machine configuration, the service type and the service volume of the request device.

Taking the cloud database CDB as an example, the adjustment interval of the integer type database parameter is described as follows:

"thread _ pool _ size": "integer", "1, 48,48] ], wherein" thread _ pool _ size "represents the adjustment interval of an integer type of database parameter, integer represents the integer type, the first value 1 is the minimum value of the parameter, the second value 48 is the maximum value of the parameter, and the third value 48 is the default parameter value (i.e. the current configuration value) of the parameter of the database to be optimized.

For database parameters of the types of character strings, enumeration and the like, the adjustment interval is composed of the following information, { parameter type, all possible discrete values, default value }, and the default value is a default parameter value (i.e. current configuration value) of the database to be optimized.

Taking a cloud database CDB as an example, explaining an adjustment interval of a default database parameter of a character string type:

"binlog _ checksum": "string", "NONE", "CRC32" "," CRC32 "", "binlog _ checksum" denotes the adjustment interval of a string-type database parameter, string refers to the string-type, the arrays of "NONE", "CRC32" are all possible discrete values of the parameter, the "CRC32" is the default parameter value, i.e. the current configuration value, of the parameter of the database to be optimized), and the value of the parameter can only be changed in discrete arrays.

It is understood that the above is only an example, in practical applications, the database manager may combine all parameters that may affect the performance of the database into a parameter list that needs to be adjusted in the above form, the adjusted parameter list is different for different database types, and some adjustments may be made for different users, which is not limited herein.

In order to improve the adjustment efficiency of the database parameters, in the embodiment of the present application, a Heuristic Algorithm (Heuristic Algorithm) is applied to the adjustment of the database parameters. The main idea is as follows: and obtaining a feasible solution of the database parameters based on a heuristic algorithm. Wherein, the heuristic algorithm has different definitions: an algorithm defined as, based on intuitive or empirical construction, giving an approximate optimal solution within acceptable computational cost (computation time, occupied space, etc.) for an instance of the optimization problem, the degree of deviation of the approximate solution from the true optimal solution not necessarily being predictable in advance; by another definition, a heuristic is a technique that allows the best solution to be found within an acceptable computational cost, but does not necessarily guarantee a feasible and optimal solution, and even in most cases does not address how close the solution is to the optimal solution. Is proposed with respect to an optimization algorithm. An optimization algorithm for a problem refers to finding the optimal solution for each instance of the problem. In some cases, especially in practical problems, the calculation time of the optimal algorithm is unbearable to people or the calculation time increases exponentially with the increase of the problem scale due to the difficulty of the problem, and at this time, only one feasible solution of the problem can be obtained through a heuristic algorithm.

Specifically, after the initial configuration information corresponding to each database parameter is obtained, at least two possible initial configuration combinations of each database parameter may be determined according to the initial configuration information. The initial configuration combination can be understood as different combinations of possible values of the database parameters. For example, assuming that the database parameters of the database to be optimized are parameter a, parameter B, and parameter C, 1000 possible values of parameter a, 500 possible values of parameter B, and 800 possible values of parameter C, at least two initial configuration combinations may be generated according to the possible values of parameter a, parameter B, and parameter C, such as initial configuration combination 1[ a₁，B₁，C₁]… …, initial configuration combination i [ A_i，B_i，C_i]Wherein A is₁、A_iRepresents the value of the parameter A, B₁、B_iRepresents the value of the parameter B, C₁、C_iRepresenting the value of the parameter C.

The initial configuration combination may include a current configuration combination of each database parameter of the database to be optimized (which may also be referred to as a default configuration combination of each database parameter of the database to be optimized), that is, the current configuration value of each database parameter of the database to be optimized may be tested to test whether it meets the performance requirement.

And then respectively performing performance test on the database to be optimized based on the obtained at least two initial configuration combinations to obtain a performance test result corresponding to each initial configuration combination, screening each initial configuration combination according to the performance test result, and if a combination with performance meeting requirements can be screened out, using the combination meeting the performance requirements as a target configuration combination, so that the requesting device can update each database parameter of the database to be optimized based on the target configuration combination.

It should be noted that, the above process is described by taking an example of the database parameter adjustment method provided in this application after the server and the requesting device interact with each other, and in practical application, the server or the requesting device may also separately complete the method. For example, the server or the requesting device initiates a parameter adjustment request, then determines the target configuration combination according to the above manner, and updates each database parameter of the data to be optimized corresponding to the server based on the obtained target configuration combination. The embodiments of the present application are not limited thereto.

In practical applications, the target configuration combination may be determined through multiple iterations, for example, in the first iteration, at least one feasible solution that satisfies the performance requirement, that is, at least one target configuration combination, may be selected from at least two initial configuration combinations based on the performance requirement of the user. In practical application, multiple iterations are usually required to obtain more and better feasible solutions, that is, various possible feasible solutions can be obtained through the second iteration, the third iteration and the like according to the above manner, and the target configuration combination of each database parameter is determined through a continuous iteration optimization manner.

For example, after the initial configuration information corresponding to each database parameter is obtained, at least two initial configuration combinations may be determined by calling the parameter list. For example, for integer type values, one can randomly take a value between the minimum and maximum values, such as the thread _ pool _ size 20, and for string type and enumeration type values, the binlog _ checksum randomly takes the value "CRC32" in the discrete parameter list. In this way, at least two sets of parameter values can be generated, each set of parameter values is, for example, { thread _ pool _ size:20, binlog _ checksum: "CRC32" … }, during a first iteration, n sets of parameter sets (i.e., initial configuration combinations) can be generated, user default parameter sets (i.e., current configuration combinations) are also added to the n sets of parameter sets, performance tests are performed on the generated n sets of parameter sets + the client default parameter sets to obtain respective corresponding performance test results, at least one feasible solution meeting performance requirements is determined according to the performance test results, the iteration is continued, a plurality of feasible solutions are determined according to the above manner, and a target configuration combination is determined from the plurality of feasible solutions.

By the embodiment of the application, after the parameter adjustment request for the database to be optimized is obtained, the initial configuration information of each database parameter of the data to be optimized can be obtained based on the request. Then, at least two possible initial configuration combinations of the database parameters are generated according to the initial configuration information. Then, based on the at least two initial configuration combinations, testing the performance of the data to be optimized to obtain a performance test result corresponding to each initial configuration combination, and then based on the performance test result, determining a target configuration combination corresponding to each database parameter, so that each database parameter of the database to be optimized can be updated based on the target configuration combination. By adopting the method, a feasible target configuration combination can be determined on the premise of ensuring the performance requirement, the problem that parameters of the database to be optimized are optimized based on a trained model through long early-stage training in the related technology is avoided, and the efficiency of optimizing the parameters of the database to be optimized is greatly improved.

The specific process of implementing tuning of the database parameters by means of iterative optimization is detailed below.

In an optional embodiment, the performing, based on the at least two initial configuration combinations, performance tests on the databases to be optimized respectively, and determining, based on a performance test result corresponding to each of the initial configuration combinations, a target configuration combination corresponding to each of the database parameters includes:

Optionally, the following operations may be repeatedly executed based on at least two initial configuration combinations in an iterative optimization manner until a preset end condition is satisfied, and a target configuration combination is determined based on each obtained initial configuration combination, where the specific process is as follows:

in the first iteration, performing performance test on each initial configuration combination to obtain a performance test result corresponding to each initial configuration combination, determining at least one initial configuration combination from the initial configuration combinations based on the performance test result corresponding to each initial configuration combination, wherein the initial configuration combination is a feasible solution of at least one database parameter meeting performance requirements, then, in order to find more feasible solutions, continuously performing iterative optimization, in the second iteration, determining at least two new configuration combinations corresponding to a new batch of database parameters from the initial configuration information, taking the at least two new configuration combinations as the at least two initial configuration combinations, and continuously determining at least one initial configuration combination corresponding to new initial configuration information in a first iteration mode, wherein the at least one initial configuration combination is not repeated here. It is understood that in practical applications, no limitation is imposed on the specific number of iterations.

According to the embodiment of the application, the initially selected configuration combination can be continuously updated in an iterative optimization mode, and then the target configuration combination is determined from the initially selected configuration combination, so that the determined target configuration combination has better performance, and the optimization of performance requirements is ensured.

the operation time length of the operation reaches a first set time length;

and receiving an operation ending instruction.

Optionally, the first set time duration may be understood as a given tuning time duration of the user, that is, the target configuration combination is determined within the tuning time duration, and then, when the first set time duration is reached, the target configuration combination may be determined from the obtained initial configuration combinations.

Alternatively, if the initially selected configuration combinations determined by at least two operations are consistent, the iteration operation may be stopped, and then the target configuration combination is determined from each initially selected configuration combination.

Or, the second set time duration may be understood as the given tuning time duration of the user, that is, the target configuration combination is determined within the tuning time duration, if the second time duration is not exceeded and the initial configuration combinations corresponding to at least two times of the above operations are consistent, the iteration operation may be stopped, and then the target configuration combination is determined from the obtained initial configuration combinations.

Or when receiving an operation ending instruction, stopping iterative operation, and determining a target configuration combination from the obtained initial configuration combinations. The operation ending instruction may be initiated by a requesting device, and is not limited herein.

According to the method and the device, different preset ending conditions can be set according to actual requirements, and when the preset ending conditions are met, the target configuration combination is determined from the obtained initial configuration combinations, so that the method is flexible and changeable, and the flexibility of determining the target configuration combination is improved.

In an optional embodiment, the determining, based on the performance test result corresponding to each initial configuration combination, an initial configuration combination from the initial configuration combinations includes:

Optionally, after the performance test result of each initial configuration combination is determined, each initial configuration combination needs to be filtered according to the performance test result and a preset performance constraint condition. If the performance test result is the performance score of the initial configuration combination and the performance constraint condition is that the performance score is not lower than a certain set threshold, the combination of which the performance score is lower than the set threshold in each initial configuration combination can be filtered. Or, the initial configuration combinations are sorted according to their performance scores in descending order, and a number of ranked combinations with a set number are filtered out, and so on, which is not limited herein.

And then, determining the initial selection configuration combination from the reserved initial configuration combinations according to the performance result corresponding to the reserved initial configuration combinations. For example, the best combination among the retained initial configuration combinations may be determined as an initial configuration combination. That is, in each iteration, a best performing initial configuration combination is determined and recorded, and then at least one target configuration combination is determined from each recorded initial configuration combination.

According to the embodiment of the application, some configuration combinations with poor performance can be removed in a filtering mode to meet the performance requirement of the database to be optimized, the configuration combinations with good performance are selected, repeated testing on the configuration combinations with poor performance can be omitted through filtering the configuration combinations with poor performance, and the efficiency is improved.

Optionally, in practical applications, in addition to considering the performance requirement (i.e., the performance constraint condition), the initial configuration combination is determined to also consider a hard requirement (i.e., the optimization constraint condition) of the user, where the hard requirement is some hard requirement of the database to be optimized.

For example, taking the CDB database as an example, the CDB database supports three forms of logging modes, namely, state, row and mixed. The statement records all related operations as a statement form of Structured Query Language (SQL). However, such a recording method cannot synchronously record some special information, such as dynamically changing values of uuid, now (), and the like. The row form is recorded on a row basis, namely, the value of each column of the related row is saved in the log, and the result is that the log file becomes very large, but the certainty of the dynamic value is guaranteed. There is also a mixed form, indicating how to log, which is determined by MySQL (MySQL _ num _ rows, indicating the number of rows in the set of results taken) itself. The performance of the state form is definitely better than the performance of the row and the mixed, but there is no way to record the dynamically changing value, if the dynamically changing value exists in the service of the user, even if part of the performance is lost, the row or the mixed needs to be selected, which belongs to the hard requirement of the user, the performance cannot be improved uniformly, and before the hard requirement of the user, the principle that the hard requirement of the user > the performance is improved needs to be followed.

The above-mentioned hard requirement is only a possible optimization constraint condition, and there are some other hard requirements, for example, in order to ensure that when the master database is down, the slave database can be switched to in time, the master-slave synchronization must be started, and in order to ensure that the data is not lost, the parameter values of the data that are timely flushed from the cache to the disk need to be started. Hard limitation can be made when generating each database parameter, and parameter values which do not accord with the service standard can not be selected.

In practical applications, the user's needs may vary, and the hard needs may be updated over time.

In practical applications, the hard requirement is very important, and when determining the target configuration combination, the hard requirement > a principle of improving performance needs to be ensured. After the hard requirements (i.e., the optimization constraints) of the user are obtained, the initial configuration combinations can be determined from the initial configuration combinations in different ways based on the hard requirements.

Mode 1: and screening according to the optimization constraint conditions, determining each initial configuration combination meeting the optimization constraint conditions from the initial configuration combinations, and then screening based on the performance constraint conditions, namely determining at least one initially selected configuration combination according to the performance test result of each initial configuration combination meeting the optimization constraint conditions. For example, assuming that the number of each initial configuration combination is 1000, the initial configuration combinations satisfying the optimization constraint condition are obtained by screening according to the optimization constraint condition, and then the number of the initial configuration combinations satisfying the optimization constraint condition is obtained by screening based on the performance constraint condition, and the number of the initial configuration combinations satisfying the performance constraint condition is obtained by 2.

Mode 2: and screening according to the performance constraint conditions, determining at least one candidate configuration combination based on the performance test result corresponding to each initial configuration combination, screening according to the optimization constraint conditions, and determining the initial selection configuration combination meeting the optimization constraint conditions from at least one candidate configuration combination. For example, assuming that the number of each initial configuration combination is 1000, the screening is performed according to the performance constraint condition, so that the number of candidate configuration combinations satisfying the performance constraint condition is 20, and then the screening is performed based on the optimization constraint condition, so that the number of initially selected configuration combinations satisfying the optimization constraint condition is 2.

It should be understood that the above description is only an example, and the embodiments of the present application are not limited in any way.

When the screening is performed through the optimization constraint conditions, the parameters in the obtained initial selection configuration combination may include parameter values hard-defined by the user, and if the user does not have the parameters hard-defined by the user, the parameter values in the initial selection configuration combination determined through the iterative operation do not need to be replaced, and the parameter values can be directly used. The specific operation is that each parameter in the initial configuration combination is matched with the hard requirement rule in turn.

According to the embodiment of the application, in practical application, the hard requirements of users can be fully considered, various possible configuration combinations of all database parameters of the database to be optimized are screened, the performance requirements and the hard requirements of the users can be simultaneously met, and the feasibility of the screened configuration combinations in practical application is ensured.

In an optional embodiment, the determining the target configuration combination based on the obtained initial configuration combinations includes any one of:

Optionally, when determining the target configuration combination of each database parameter of the database to be optimized, there may be multiple possible implementation manners, which are specifically as follows:

according to the recorded performance test result of the initially selected configuration combination obtained by each iteration, one target configuration combination with the best performance is selected as the final configuration combination of each database parameter of the database to be optimized, or at least two target configuration combinations with the highest performance ranking are sent to the request device, and the user corresponding to the request device selects the final configuration combination of each database parameter of the database to be optimized, which is not limited herein.

The initially selected configuration combination obtained from each iteration can be used as the target configuration combination. For example, in practical applications, if the database adjustment time (e.g., the first set time length) of the user is relatively urgent, each database parameter of the database to be optimized may be adjusted in a manner of optimizing while using, for example, an optimal initial configuration combination (i.e., an optimal configuration combination that satisfies both the performance constraint condition and the optimization constraint condition) obtained by each iteration may be sent to the requesting device, if the adjustment time is not reached, the iteration may be continued, if the initial configuration combination obtained by the current iteration is more optimal, the initial configuration combination is sent to the requesting device, and if the initial configuration combination obtained by the previous iteration is not more optimal, the iteration is continued until the adjustment time is reached.

And determining at least one target configuration combination from the initial configuration combinations according to the optimization constraint conditions and the performance constraint conditions of the database to be optimized. For example, the recorded performance test results of the primarily selected configuration combinations obtained by each iteration may be screened according to the performance constraint conditions, at least one target configuration combination satisfying the optimization constraint conditions is screened from the performance constraint conditions, the at least one target configuration combination is sent to the requesting device, and the user corresponding to the requesting device performs screening.

It should be understood that the above description is only an example, and the embodiments of the present application are not limited thereto.

According to the method and the device, the target configuration combination can be determined in various modes, the modes are flexible and various, and the flexibility of the mode for determining the configuration combination of each database parameter of the database to be optimized is improved.

The following is a detailed description of how performance testing is performed.

In an optional embodiment, the performing, based on the at least two initial configuration combinations, performance tests on the databases to be optimized respectively includes:

Alternatively, the database instance is comprised of various cache pools and background processes. Databases are organized, shareable collections of data that are stored long term in a computer. The data in the database is organized, described and stored together in a certain data model, has the characteristics of minimal redundancy, high data independence and expansibility and can be shared by a plurality of users within a certain range. The database instance is composed of an operating system background process and a distributed memory area, and is a set of a memory and a background process. It should be noted that one database instance may be used for one database, and multiple database instances may also be used for one database at the same time, the relationship between the database instance and the database is a one-to-many relationship, and the database instance is responsible for maintaining and accessing the data in the database.

During performance testing, each initial configuration combination can be applied to a database instance of the database to be optimized, and the performance testing is performed on the database instance corresponding to each initial configuration combination to obtain a performance testing result corresponding to each initial configuration combination.

In practical applications, during performance testing, performance testing information including a plurality of performance parameters (such as throughput, delay, etc.) may be obtained, and for a user, only one or more performance parameters of the performance parameters may be concerned, so that the performance parameters of interest to the user need to be screened out. For example, the screening may be performed by regular expressions. The regular expression can match data meeting the content required by the user from each data record (namely, a performance vector group) in a mode of pattern matching, and then the data meeting the requirement and in a fixed format is obtained. For example, a pattern matching grammar can be written to obtain only the data that meets the requirement, and in the regular expression, a single character string is used to describe and match a series of character strings matching a certain syntactic rule. For example, [ \ s ] represents a character string, [ \ w ] represents a character, a number, and an underline. Taking a regular expression of \ s \/95 th percentile [ \ s ] (\ d + \\ \ d +) as an example, the regular expression indicates that any character string +95th percentile: character string + any character string + number is to be acquired, and recording the data conforming to the regular expression can filter the data to acquire certain desired data, such as throughput and delay of the data.

According to the embodiment of the application, the performance test can be performed by applying each initial configuration combination to the database instance, so that the efficiency of the performance test is improved.

In a possible embodiment, the performing a performance test on the database instance corresponding to each of the initial configuration combinations includes:

acquiring historical service data of the database to be optimized;

Optionally, the historical service data may be understood as a real load in a service scenario of the user, a test case is generated through the real load, and a performance test is performed on the database instance corresponding to each initial configuration combination based on the test case to obtain a corresponding performance test result. In practical application, the performance test may be performed without using the test case generated by the historical service data, which is not limited herein.

For example, a historical service scenario request record of a user may be obtained, and the request record of the user is analyzed, where the analysis content mainly includes: the amount of data stored (e.g., how many databases, how many tables, how much data is in each database table), the type of record requested (e.g., query, insert, delete, update), the amount requested per minute or per second, and the type of query (e.g., whether the query is a simple query or a complex query). And analyzing to obtain a corresponding test case, simulating a request scene of the real service of the user, and taking the request as a database batch request (namely the test case). The requests are put on a database instance for testing to obtain performance indexes which can be processed by the database, such as the acceptable request quantity per second, the transaction number per second and the transaction delay, and the performance indexes are processed to obtain a performance index vector group (also called as a performance vector group (namely the performance test information described above).

By the aid of the method and the device, the database instance of the database to be optimized can be subjected to performance test according to the test case generated by actual historical service data, a performance test result is obtained, actual service requirements of users can be met by the method, and practicability is improved.

Optionally, when performing the performance test, the performance test may be performed in a parallelized manner using at least two test devices. For example, assuming that n sets of initial configuration combinations (which may also be referred to as n sets of initialization parameter sets or n sets of parameter sets) are generated, the n sets of parameter sets may be distributed to m machines (i.e., test devices), the number of parameter sets held by each machine is n/m, each machine holds n/m parameter sets in a state where each machine is healthy, each machine concurrently applies the parameters held by itself to a plurality of database instances, and a performance test is performed using the real load in the user normal business scenario to obtain a performance index of each parameter set. Collecting the performance indexes of each parameter group, eliminating the performance indexes by utilizing a heuristic algorithm, generating a new parameter group (namely at least two new configuration combinations described in the foregoing), recording the new parameter group and the corresponding performance indexes, and recording the parameter group as an initial selection configuration combination if a parameter group with a better performance index is found. Wherein n and m are positive integers, and n is greater than or equal to m.

Optionally, during performance testing, each testing device and health monitoring are required. The specific implementation mode is that heartbeat information of each testing device is received, the heartbeat information is information for verifying whether the testing device can normally work or not, whether each testing device is in a normal working state or not is determined according to the heartbeat information of each testing device, and if one or more testing devices are not in the normal working state, configuration combinations issued to the one or more testing devices are recovered and are re-issued to other testing devices in the normal working state.

Among them, a Heartbeat Message (Heartbeat Message) is a Message that a sending source (test device) sends to a receiving source (e.g., a server) and allows the receiving source to determine if and when the sending source fails or terminates. Generally, heartbeat messages are sent from the time the sender is started until the sender is turned off, during which the sender sends messages periodically or repeatedly without interruption. When the receiver does not receive the message within a certain message receiving period, the receiver may consider that the transmission source has been turned off, has failed, and is currently unavailable. Heartbeat messages may often be used for high availability or fault tolerant processing purposes.

Specifically, if the test device is processing a transaction, the test device sends a status message to the server at regular intervals, for example, 5 seconds, such as "processing a transaction", if the test device does not send verification information to the server after 5 seconds, the server sends a message requesting the status message to the slave machine, and if the verification message is not received within 5 seconds, the server may think that the test device is down or cannot provide normal service. If it is assumed that the initial configuration combination is n groups and the number of the test devices is m, if it is found that a certain test device is down or cannot provide normal service, the server will equally distribute the parameter group (i.e. the initial configuration combination) allocated to the test device to the other m-1 test devices.

The above is merely an example, and the embodiments of the present application are not limited herein.

Through the embodiment of the application, the performance test can be carried out in a parallelization test mode and a health monitoring mode, on one hand, the efficiency of the performance test is improved, on the other hand, the performance test can be carried out by using the normally working test equipment, and the problem that the performance test cannot be carried out due to the fact that the test equipment is down or cannot provide normal service is avoided.

Optionally, performance testing may be performed on the to-be-optimized database based on each initial configuration combination, and performance testing information (i.e., the performance vector group described above) corresponding to each initial configuration combination is obtained, where the performance testing information may be determined according to an actual requirement, that is, may be determined according to which aspect of performance requirements is actually required to be met. Such as the performance test information may include at least one of test throughput or test delay. Where throughput and delay may be performance parameters.

The throughput refers to the number of requesting devices corresponding to the database to be optimized, which can successfully transmit data in unit time (which can be measured by bits, bytes, packets, etc.).

Then, current configuration information of each database parameter of the database to be optimized is obtained, performance test is carried out on the data to be optimized based on the current configuration information, and performance benchmark information corresponding to the database to be optimized is obtained, wherein the performance benchmark information can comprise at least one of benchmark throughput or benchmark delay.

Then, for each initial configuration combination, a performance test result corresponding to the initial configuration combination can be determined according to the performance benchmark information of the database to be optimized and the performance test information corresponding to each initial configuration combination.

For example, assuming that the performance test information includes test throughput and the performance benchmark information includes benchmark throughput, a throughput performance test result may be determined based on the test throughput and the benchmark throughput, and a specific calculation manner may be: taking the difference between the test throughput and the reference throughput as a first difference, and taking the ratio between the first difference and the reference throughput as the throughput performance test result, for example, for the ith initial configuration combination, the test throughput corresponding to the initial configuration combination is T_iThe base throughput of the database to be optimized is T_DAnd the throughput performance test result is s, then the throughput performance test result corresponding to the ith initial configuration combination may be determined according to the following formula:

assuming that the performance test information includes a test delay and the performance reference information includes a reference delay, the delay performance test result may be determined based on the test delay and the reference delay, and the specific calculation manner may be: taking the difference between the test delay and the reference delay as a second difference, and taking the ratio between the second difference and the reference delay as the delay performance test result, for example, for the ith initial configuration combination, the test delay corresponding to the initial configuration combination is L_iThe base delay of the database to be optimized is L_DAnd the delay performance test result is s, then the delay performance test result corresponding to the ith initial configuration combination can be determined according to the following formula:

assuming that the performance test information includes test throughput and test delay, and the performance benchmark information includes test delay and benchmark delay, the performance test result corresponding to the ith initial configuration combination may be determined according to the following formula:

it is understood that the above is only an example, and in the embodiment of the present application, no limitation is made to the specific formula of the performance test result.

According to the embodiment of the application, the performance test result can be determined based on actual performance requirements, the initial configuration combination with poor performance is eliminated through the performance test result, and a foundation is laid for finally determining the configuration combination with good performance by continuously eliminating the configuration combination with poor performance.

In an alternative embodiment, the performance test information includes test throughput and test delay, and the performance benchmark information includes benchmark throughput and benchmark delay;

Optionally, the factor of the ratio of different performance parameters (such as throughput and delay) may also be considered when determining the performance test result.

Among the user requirements, some users may pay more attention to throughput, some users may pay more attention to delay, and the ratio of the first weight corresponding to throughput and the second weight corresponding to delay may be set according to the performance requirements of different users, that is, the ratio of the first weight and the second weight needs to be adapted to the user requirements.

When the performance test information includes test throughput and test delay, and the performance reference information includes reference throughput and reference delay, the performance test result corresponding to the ith initial configuration combination may be determined according to the following formula:

where s is the performance test result, and for the ith initial configuration combination, T_iCombining corresponding test throughputs, T, for initial configuration_DFor the base throughput of the database to be optimized, L_iCombining corresponding test delays, L, for the initial configuration_DFor the base latency of the database to be optimized, α is the first weight and β is the second weight.

For each initial configuration combination, a performance test result (also referred to as a score) s of each initial configuration combination can be obtained through formula calculation, s represents the quality of the ith initial configuration combination, and a parameter group with a low score (i.e. the initial configuration combination) can be eliminated through the score of each initial configuration combination. Selecting high-score parameter set, and then generating new parameter set from some parameters selected from high-score parameter set, such as parameter set 1 containing parameter k₁，k₂，k₃Parameter set 2 is m₁，m₂，m₃The new parameter set 3 generated may be k₁，m₂，m₃H, may also be k₁，k₂，m₃Randomly moving each parameter of each parameter group into a new parameter group with a certain probability to generate a new parameter group, wherein k is_iAnd m_iIs a parameter value. And (3) eliminating some parameter groups with low scores in each iteration, generating some new parameter groups, and as the number of iterations increases, if a parameter group with a score higher than the highest history is found in each iteration, keeping and recording the parameter groups.

In practical applications, in addition to determining a new parameter set according to a reserved parameter set (i.e. some parameter sets with high scores), the new parameter set may also be determined according to the initial configuration information, or the new parameter set may be determined according to the reserved parameter set and the initial configuration information, which is not limited herein.

By the embodiment of the application, the performance test result can be determined according to the occupied proportion of different performance parameters, and by adopting the mode, the determined performance test result can be adapted to the service requirement of a user, so that the adaptability to the service requirement is improved.

The following description will be given of a specific implementation manner of the database parameter adjustment method in the embodiment of the present application, taking an example that a user initiates a parameter adjustment request through a client and processes the parameter adjustment request through a server, where the server may include a control end (e.g., a controller, an electronic device such as a server and a computer) and a management end (e.g., a Master, an electronic device such as a server and a computer), and the specific implementation manner is as follows:

referring to fig. 3, fig. 3 is a schematic diagram of an overall framework of a database parameter adjustment method provided in an embodiment of the present application, as shown in fig. 3, a user initiates a parameter adjustment request at a client, a server control end receives and responds to the parameter adjustment request, obtains initial configuration information of each database parameter of a database to be optimized, generates an initialization parameter set (i.e., at least two initial configuration combinations) required by a heuristic algorithm based on the initial configuration information, submits the at least two initialization parameter sets to a management end, the management end applies the initialization parameter set to a database instance, and uses a real load under a user normal service scenario as a test load (i.e., a test case), performs a pressure test on each initialization parameter set, obtains a performance vector group (including throughput and delay) corresponding to each initialization parameter set, where the performance vector group is the above performance test information, and eliminating and selecting the parameter group of the iteration through the performance vector group to obtain an initial selection configuration combination, updating to obtain a new initialization parameter group corresponding to the next iteration, and continuing to iterate according to the process. And taking the parameter with the best performance obtained by each iteration as a primary configuration combination, recording the primary configuration combination into a file or a database, and sending the file or the database to a control end for judgment to determine whether to recommend the primary configuration combination as a target configuration combination to a user.

The database corresponding to the database example shown in fig. 3 may be a DB database, or may be a CDB database, which is not limited herein.

As an optional mode, the device corresponding to the database parameter adjustment method of the present application may have a structure shown in fig. 4, refer to fig. 4, and fig. 4 is a schematic structural diagram of a database parameter adjustment device provided in an embodiment of the present application, and as shown in fig. 4, the device includes a load generation module, a load playback module, an initialization parameter generation module, a heuristic algorithm training module, a sexual vector group acquisition module, and a parameter group recommendation module.

The load generation module is used for generating a test load based on the real load in the normal service scene.

And the load playback module is used for carrying out pressure test on each initialization parameter group based on the obtained test load.

And the initialization parameter generation module is used for responding to the parameter adjustment request, acquiring initial configuration information of each database parameter of the database to be optimized, and generating an initialization parameter group based on the initial configuration information.

And the heuristic algorithm training module is used for updating, eliminating and selecting each initialization parameter group of each iteration based on the initialization parameter group to generate a new initialization parameter group of the next iteration.

And the characteristic vector group acquisition module is used for acquiring the performance vector group corresponding to each initialization parameter group.

And the parameter set recommendation module is used for determining a target configuration combination recommended to the user based on the primary selection configuration combination generated by each iteration.

The specific operation principle of the management end is described in detail below.

Referring to fig. 5, fig. 5 is a schematic diagram of an operating principle of a management end according to an embodiment of the present application, as shown in fig. 5, in the management end, in a parallelization manner, a Master of the management end distributes n initialization parameter sets generated by an initialization parameter generation module, and distributes the parameters to at least one sub-machine (i.e., the testing device described above), each sub-machine receives at least one initialization parameter set belonging to itself, performs multi-database pressure testing, returns a performance vector set of each initialization parameter set (i.e., a full performance vector set shown in the figure), is uniformly managed by the Master, performs updating, elimination and selection of the initialization parameter sets, generates a new generation of initialization parameter sets, generates a new parameter set, and then determines whether a parameter set obtained in a current iteration is better than a parameter set obtained in a previous iteration, if the optimal result is better, the optimal result is transmitted to the client, and if the optimal result is not better, the updating iteration is continued.

When the performance test is carried out, after each database instance corresponding to the database to be optimized configures parameters according to each initialization parameter group, the database is waited to be restarted, the performance test is carried out, pressure test files of different test benchmarks (namely, the performance test is carried out according to different initialization parameter groups) are analyzed through a regular expression, throughput and delay information can be obtained, a performance vector group is obtained, the performance vector group is stored and transmitted to a management terminal, each sub-machine carries out uniform distribution on the parameters issued by the management terminal again, and a plurality of database instances are subjected to pressure test by utilizing a multi-process technology, so that the test efficiency is improved.

Next, the operation of each sub-machine for performance testing will be described, referring to fig. 6, fig. 6 is a schematic diagram of the operation of the sub-machine for performance testing provided by the embodiment of the present application,

for each sub-machine, the sub-machine can be responsible for performance testing of at least one database instance, at this time, based on the number of database instances that the sub-machine can simultaneously perform parallel testing, at least one initialization parameter set is allocated to the sub-machine, then the at least one initialization parameter set is applied to the at least one database instance through the sub-machine, performance testing is performed, at least one corresponding performance vector set is obtained through analysis, the performance vector sets corresponding to the initialization parameter sets are integrated, and the integrated performance vector sets are uniformly sent to the management terminal. Fig. 6 is a schematic diagram of a sub-machine performing performance tests on three database instances, which is not limited in this embodiment of the present application.

In the process of performing the performance test, there may be a situation that one or more sub-machines cannot be served, and in order to ensure that the performance test is performed smoothly, the health monitoring of the sub-machines may be performed. Referring to fig. 7, fig. 7 is a schematic view illustrating a process of updating and eliminating a parameter set by a health monitoring mode according to an embodiment of the present application, and as shown in fig. 7, a Master has an independent thread, monitors the health status of each sub-machine according to acquired heartbeat information of each sub-machine, and if a certain sub-machine has a problem, the Master recovers the parameter sent to the sub-machine and sends the parameter to other healthy sub-machines, so as to ensure high availability and horizontal expansion capability. And issuing the initialization parameter group generated by each iteration to each sub-machine, collecting and obtaining a performance vector group corresponding to each initialization parameter group through database example pressure measurement, after each iteration is finished, eliminating and selecting through a performance test result obtained by the performance vector group, transmitting the best parameter group in the current iteration to a control end, and judging whether to recommend the parameter group by the control end, if so, determining whether to recommend the parameter group to a client side by judging whether the parameter group meets the hard requirement of a user.

In order to ensure that the performance test is more adaptive to the service scenario of the user, the performance test may also be performed based on the real load of the service scenario of the user, referring to fig. 8, where fig. 8 is a schematic flow chart of performing the performance test through the real load according to an embodiment of the present application, and as shown in fig. 8, historical data of the real load of the service scenario of the user is input, a request record under the real load of the user is analyzed, and the analysis content mainly includes: the amount of data stored (e.g., how many databases, how many tables, how much data is in each database table), the type of record requested (e.g., query, insert, delete, update), the amount requested per minute or per second, and the type of query (e.g., whether the query is a simple query or a complex query). Simulating a request scene of real service of a user, taking the request as a database batch request, and generating a corresponding user request script (namely a test case). The user request scripts are applied to the database example for performance testing, and as shown in the figure, the performance testing is performed through the database example 1+ parameter group 1, the database example 2+ parameter group 2, and the database example 3+ parameter group 3, so as to obtain the performance vector group corresponding to each parameter group, such as the performance vector group 1 corresponding to the parameter group 1, the performance vector group 2 corresponding to the parameter group 2, and the performance vector group 3 corresponding to the parameter group 3 shown in the figure.

As shown in fig. 9, in the above manner, the Manager may select and obtain the best parameter set (i.e., the initially selected configuration combination) in each iteration through a heuristic algorithm, and the control end performs rule matching to check whether the hard requirement of the client is satisfied (i.e., whether the determination shown in the figure satisfies the optimization constraint condition), if not, starts a new generation of iteration and selection, if so, compares the best parameter set with the previously recommended parameter set, and if so, means that the best parameter set retained in the previous iteration is still the best parameter set for the iteration, the recommendation need is not repeated, and if the performance of the parameter set obtained in the current iteration is better, and recommending the parameters to the client again, wherein if the effect of the parameter group recommended before is not good, the parameter group generated by the iteration is not required to be recommended.

After receiving the parameter group recommended by the control end, the client may update each database parameter of the database to be optimized of the client based on the recommended parameter group.

It should be noted that the database parameter adjustment method in the embodiment of the present application may be applied to various types of databases, such as a key value database, a relational database, and the like, and only the test load (i.e., each database parameter of the database to be optimized) and the new parameter search space (i.e., the initial configuration information) need to be changed. In a small company, the requirement of recruiting a deep database manager is not met, the technical scheme of the embodiment of the application reduces the learning threshold of database parameter adjustment, and a better parameter set can be found under most conditions, so that most parameter adjustment requirements can be met. In a large-scale company, when the scheme faces a new load type or a new database type, the scheme can still provide an auxiliary parameter adjusting scheme for a senior database administrator, and the senior database administrator can find a better parameter set according to own experience and the database parameter adjusting method provided by the embodiment of the application.

By the embodiment of the application, the parameter adjusting mode provided by the embodiment of the application is tested through different testing environments, and compared with the performance of the current default configuration combination of the data to be optimized, the throughput is remarkably improved, and the delay is reduced. In addition, the time for automatically adjusting the database parameters is shortened to an hour level from days, and a better performance parameter group is provided for a user under the condition of high time requirement so as to meet the database performance requirement of the user and greatly improve the adjustment efficiency.

Each electronic device (including but not limited to a server, a user terminal, a management end, a control end, and a client) involved in the embodiments of the present application may be a node of a blockchain system, a server (including but not limited to a server) involved may be a blockchain server, and data required to be stored in the embodiments of the present application may also be stored in the blockchain node. The database parameter adjustment system according to the embodiment of the present application may be a distributed system formed by connecting a user terminal (e.g., a client, a user terminal), a plurality of blockchain nodes (any type of computing devices in an access network, e.g., a server, a user terminal) through a network communication manner.

Taking a distributed system as an example of a blockchain system, referring To fig. 10, fig. 10 is an optional structural schematic diagram of the distributed system 100 applied To the blockchain system provided in this embodiment of the present application, and is formed by a plurality of nodes 200 (computing devices in any form in an access network, such as servers and user terminals) and a user end 300, a Peer-To-Peer (P2P, Peer To Peer) network is formed between the nodes, and the P2P Protocol is an application layer Protocol operating on a Transmission Control Protocol (TCP). In a distributed system, any machine, such as a server or a terminal, can join to become a node, and the node comprises a hardware layer, a middle layer, an operating system layer and an application layer.

Referring to the functions of each node in the blockchain system shown in fig. 10, the functions involved include:

1) routing, a basic function that a node has, is used to support communication between nodes.

Besides the routing function, the node may also have the following functions:

2) the application is used for being deployed in a block chain, realizing specific services according to actual service requirements, recording data related to the realization functions to form recording data, carrying a digital signature in the recording data to represent a source of task data, and sending the recording data to other nodes in the block chain system, so that the other nodes add the recording data to a temporary block when the source and integrity of the recording data are verified successfully.

For example, the services implemented by the application include:

2.1) wallet, for providing the function of transaction of electronic money, including initiating transaction (i.e. sending the transaction record of current transaction to other nodes in the blockchain system, after the other nodes are successfully verified, storing the record data of transaction in the temporary blocks of the blockchain as the response of confirming the transaction is valid; of course, the wallet also supports the querying of the remaining electronic money in the electronic money address;

and 2.2) sharing the account book, wherein the shared account book is used for providing functions of operations such as storage, query and modification of account data, record data of the operations on the account data are sent to other nodes in the block chain system, and after the other nodes verify the validity, the record data are stored in a temporary block as a response for acknowledging that the account data are valid, and confirmation can be sent to the node initiating the operations.

2.3) Intelligent contracts, computerized agreements, which can enforce the terms of a contract, implemented by codes deployed on a shared ledger for execution when certain conditions are met, for completing automated transactions according to actual business requirement codes, such as querying the logistics status of goods purchased by a buyer, transferring the buyer's electronic money to the merchant's address after the buyer signs for the goods; of course, smart contracts are not limited to executing contracts for trading, but may also execute contracts that process received information.

3) And the Block chain comprises a series of blocks (blocks) which are mutually connected according to the generated chronological order, new blocks cannot be removed once being added into the Block chain, and recorded data submitted by nodes in the Block chain system are recorded in the blocks.

Referring to fig. 11, fig. 11 is an optional schematic diagram of a Block Structure (Block Structure) provided in this embodiment, where each Block includes a hash value of a transaction record stored in the Block (hash value of the Block) and a hash value of a previous Block, and the blocks are connected by the hash value to form a Block chain. The block may include information such as a time stamp at the time of block generation. A block chain (Blockchain), which is essentially a decentralized database, is a string of data blocks associated by using cryptography, and each data block contains related information for verifying the validity (anti-counterfeiting) of the information and generating a next block.

Referring to fig. 12, fig. 12 is a schematic structural diagram of another database parameter adjustment apparatus according to an embodiment of the present application. The database parameter adjustment device 1 provided in the embodiment of the present application includes:

a parameter adjustment request obtaining module 11, configured to obtain a parameter adjustment request for a database to be optimized;

an initial configuration information obtaining module 12, configured to obtain initial configuration information corresponding to each database parameter of the database to be optimized according to the parameter adjustment request;

a configuration combination determining module 13, configured to determine at least two initial configuration combinations corresponding to the database parameters according to the initial configuration information corresponding to the database parameters;

a target configuration parameter determining module 14, configured to perform performance tests on the databases to be optimized based on the at least two initial configuration combinations, and determine a target configuration combination corresponding to each database parameter based on a performance test result corresponding to each initial configuration combination, so as to update each database parameter of the databases to be optimized based on the target configuration combination.

the operation time length of the operation reaches a first set time length;

and receiving an operation ending instruction.

acquiring historical service data of the database to be optimized;

In the embodiment of the application, after a parameter adjustment request for a database to be optimized is obtained, initial configuration information of each database parameter of the data to be optimized can be obtained based on the request. Then, at least two possible initial configuration combinations of the database parameters are generated according to the initial configuration information. Then, based on the at least two initial configuration combinations, testing the performance of the data to be optimized to obtain a performance test result corresponding to each initial configuration combination, and then based on the performance test result, determining a target configuration combination corresponding to each database parameter, so that each database parameter of the database to be optimized can be updated based on the target configuration combination. By adopting the method, a feasible target configuration combination can be determined on the premise of ensuring the performance requirement, the problem that parameters of the database to be optimized are optimized based on a trained model through long early-stage training in the related technology is avoided, and the efficiency of optimizing the parameters of the database to be optimized is greatly improved.

In a specific implementation, the database parameter adjusting apparatus 1 may execute the implementation manners provided in the steps in fig. 2 through the built-in function modules, which may specifically refer to the implementation manners provided in the steps, and are not described herein again.

The database parameter adjusting method in the present application is mainly described above as being implemented by hardware, but the execution subject of the database parameter adjusting method in the present application is not limited to hardware, and the execution subject of the database parameter adjusting method in the present application may also be software, and the database parameter adjusting device may be a computer program (including program code) running in a computer device, for example, the database parameter adjusting device is an application software; the apparatus may be used to perform the corresponding steps in the methods provided by the embodiments of the present application.

In some embodiments, the database parameter adjusting apparatus provided in the embodiments of the present Application may be implemented by a combination of hardware and software, and by way of example, the database parameter adjusting apparatus provided in the embodiments of the present Application may be a processor in the form of a hardware decoding processor, which is programmed to execute the database parameter adjusting method provided in the embodiments of the present Application, for example, the processor in the form of the hardware decoding processor may employ one or more Application Specific Integrated Circuits (ASICs), DSPs, Programmable Logic Devices (PLDs), Complex Programmable Logic Devices (CPLDs), Field Programmable Gate Arrays (FPGAs), or other electronic components.

In other embodiments, the database parameter adjusting apparatus provided in this embodiment may be implemented in a software manner, and the database parameter adjusting apparatus 1 shown in fig. 12 may be software in the form of a program, a plug-in, and the like, and includes a series of modules, including a parameter adjustment request obtaining module 11, an initial configuration information obtaining module 12, a configuration combination determining module 13, and a target configuration parameter determining module, for implementing the database parameter adjusting method provided in this embodiment.

Referring to fig. 13, fig. 13 is a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in fig. 13, the electronic device 1000 in the present embodiment may include: the processor 1001, the network interface 1004, and the memory 1005, and the electronic device 1000 may further include: a user interface 1003, and at least one communication bus 1002. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display) and a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a standard wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1004 may be a high-speed RAM memory or a non-volatile memory (e.g., at least one disk memory). The memory 1005 may optionally be at least one memory device located remotely from the processor 1001. As shown in fig. 13, a memory 1005, which is a kind of computer-readable storage medium, may include therein an operating system, a network communication module, a user interface module, and a device control application program.

In the electronic device 1000 shown in fig. 13, the network interface 1004 may provide a network communication function; the user interface 1003 is an interface for providing a user with input; and the processor 1001 may be used to invoke computer programs stored in the memory 1005.

It should be understood that in some possible embodiments, the processor 1001 may be a Central Processing Unit (CPU), and the processor may be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The memory may include both read-only memory and random access memory, and provides instructions and data to the processor. The portion of memory may also include non-volatile random access memory. For example, the memory may also store device type information.

In a specific implementation, the electronic device 1000 may execute the implementation manners provided in the steps in fig. 2 through the built-in functional modules, which may specifically refer to the implementation manners provided in the steps, and are not described herein again.

An embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and is executed by a processor to implement the method provided in each step in fig. 2, which may specifically refer to the implementation manner provided in each step, and is not described herein again.

The computer readable storage medium may be an internal storage unit of the task processing device provided in any of the foregoing embodiments, for example, a hard disk or a memory of an electronic device. The computer readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, a Smart Memory Card (SMC), a Secure Digital (SD) card, a flash card (flash card), and the like, which are provided on the electronic device. The computer readable storage medium may further include a magnetic disk, an optical disk, a read-only memory (ROM), a Random Access Memory (RAM), and the like. Further, the computer readable storage medium may also include both an internal storage unit and an external storage device of the electronic device. The computer-readable storage medium is used for storing the computer program and other programs and data required by the electronic device. The computer readable storage medium may also be used to temporarily store data that has been output or is to be output.

Embodiments of the present application provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the method provided by any one of the possible embodiments of fig. 2.

The terms "first", "second", and the like in the claims and in the description and drawings of the present application are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus. Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments. The term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

The above disclosure is only for the purpose of illustrating the preferred embodiments of the present application and is not to be construed as limiting the scope of the present application, so that the present application is not limited thereto, and all equivalent variations and modifications can be made to the present application.

Claims

1. A database parameter adjustment method is characterized by comprising the following steps:

acquiring a parameter adjustment request for a database to be optimized;

determining at least two initial configuration combinations corresponding to each database parameter according to the initial configuration information corresponding to each database parameter;

2. The method according to claim 1, wherein the performing, based on the at least two initial configuration combinations, performance tests on the database to be optimized, and determining, based on a performance test result corresponding to each of the initial configuration combinations, a target configuration combination corresponding to each of the database parameters includes:

determining an initial configuration combination from the initial configuration combinations based on a performance test result corresponding to each initial configuration combination;

3. The method of claim 2, wherein determining an initial configuration combination from the initial configuration combinations based on the performance test result corresponding to each initial configuration combination comprises:

4. The method according to claim 2 or 3, wherein the determining an initial configuration combination from the initial configuration combinations based on the performance test result corresponding to each initial configuration combination comprises:

obtaining an optimization constraint condition corresponding to the database to be optimized;

determining each initial configuration combination meeting the optimization constraint condition in each initial configuration combination, and determining an initial selection configuration combination according to a performance test result of each initial configuration combination meeting the optimization constraint condition;

5. The method according to claim 2 or 3, wherein the preset end condition comprises at least one of:

the operation time length of the operation reaches a first set time length;

the operation time length of the operation does not exceed a second set time length, and the initial selection configuration combinations corresponding to at least two times of the operation are consistent;

and receiving an operation ending instruction.

6. The method according to claim 2 or 3, wherein the determining the target configuration combination based on the obtained initial configuration combinations comprises any one of:

taking each initially selected configuration combination as a target configuration combination;

7. The method according to claim 1, wherein the performing performance tests on the databases to be optimized based on the at least two initial configuration combinations respectively comprises:

and respectively applying each initial configuration combination to a database instance of the database to be optimized, and carrying out performance test on the database instance corresponding to each initial configuration combination.

8. The method of claim 7, wherein performing the performance test on the database instance corresponding to each of the initial configuration combinations comprises:

acquiring historical service data of the database to be optimized;

9. The method according to claim 1, wherein the performing performance tests on the databases to be optimized based on the at least two initial configuration combinations respectively comprises:

and distributing the at least two initial configuration combinations to at least two test devices, so as to perform performance test on the database to be optimized in parallel through the at least two test devices based on the respectively distributed initial configuration combinations.

10. The method according to claim 1, wherein the performing performance tests on the databases to be optimized based on the at least two initial configuration combinations respectively comprises:

and for each initial configuration combination, determining a performance test result corresponding to the initial configuration combination according to the performance benchmark information and the performance test information corresponding to the initial configuration combination.

11. The method of claim 10, wherein the performance test information comprises test throughput and test delay, and wherein the performance benchmark information comprises benchmark throughput and benchmark delay;

for each initial configuration combination, determining a performance test result corresponding to the initial configuration combination according to the performance benchmark information and the performance test information corresponding to the initial configuration combination includes:

12. A database parameter adjustment apparatus, comprising:

the initial configuration information acquisition module is used for acquiring initial configuration information corresponding to each database parameter of the database to be optimized according to the parameter adjustment request;

13. An electronic device comprising a processor and a memory, the processor and the memory being interconnected;

the memory is used for storing a computer program;

the processor is configured for performing the method as claimed in any one of claims 1 to 11 when the computer program is invoked.

14. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which is executed by a processor to implement the method as claimed in any one of claims 1 to 11.