CN108920722B - Parameter configuration method and device and computer storage medium - Google Patents
Parameter configuration method and device and computer storage medium Download PDFInfo
- Publication number
- CN108920722B CN108920722B CN201810864294.0A CN201810864294A CN108920722B CN 108920722 B CN108920722 B CN 108920722B CN 201810864294 A CN201810864294 A CN 201810864294A CN 108920722 B CN108920722 B CN 108920722B
- Authority
- CN
- China
- Prior art keywords
- connection
- concurrent
- time
- connections
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 230000004083 survival effect Effects 0.000 claims description 29
- 238000004458 analytical method Methods 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000007619 statistical method Methods 0.000 claims description 7
- 239000000126 substance Substances 0.000 claims description 3
- 238000010845 search algorithm Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
Images
Abstract
A method, apparatus, and computer storage medium for parameter configuration, comprising: acquiring start-stop time information of each connection within a preset time length; and according to the acquired start-stop time information, performing parameter configuration of the connection pool. According to the embodiment of the invention, the configuration adjustment of the connection pool is realized according to the application requirement, and the application efficiency of database connection is improved; furthermore, parameter configuration of the connection pool is carried out by calling an Application Programming Interface (API) interface, so that configuration efficiency of the connection pool and working efficiency of application are improved.
Description
Technical Field
The present disclosure relates to, but not limited to, database technologies, and more particularly, to a method, an apparatus, and a computer storage medium for parameter configuration.
Background
In world wide Web (Web) based database applications, such as large e-commerce websites, there is a large amount of concurrent access online; if the database connection is established once for each Web request, the database connection needs to be established and closed frequently, which results in large time and resource consumption, reduced system performance, and system crash in severe cases. The connection pool can solve the problems, and the realization principle is as follows: the method comprises the steps of establishing a certain number of database connections (hereinafter referred to as connections) in advance, placing the database connections into a buffer pool, taking out required connections from the buffer pool when a connection request is received, and establishing new connections when no available connections exist in the buffer pool; and after the connection is used, the data is placed back into the buffer pool again for the next connection request. And the connection in the buffer pool is reused, so that the system overhead is reduced to a certain extent.
Parameter configuration is an important component of the application database connection pool, and in the related art, parameter configuration is generally performed by establishing a configuration table of extensible markup language (XML). Wherein the connection number includes initial connection number (initial Szie), maximum active connection number (maxActive), minimum idle connection number (minide), and maximum idle connection number (maxIdle); wherein, the active connection refers to the connection which is distributed from the connection pool and is in use, and the idle connection refers to the connection which is established in the connection pool but is not distributed yet; among the above parameters, the initial connection number plays a role in initializing the connection pool; the maximum active connection number is used for limiting the maximum connection number allowed to be created in the connection pool, and when the connection request number exceeds the maximum active connection number, the exceeding part of connection requests are added into the waiting queue; if there is no database connection available within a preset maximum wait period, the system will throw an exception. The minimum idle connection number and the maximum idle connection number control the number of idle connections which can be maintained in the connection pool, and after the connection taken out from the connection pool is used, the connection can be returned to the connection pool by executing a closing method. When the number of the reclaimed connections exceeds the value of the maximum number of the idle connections, the redundant connections are closed by the connection pool. In the management of the connection pool, maintenance of the number of idle connections in the connection pool is critical. Maintaining a certain number of idle connections in the connection pool helps to provide response speed of the application system, but if the number of idle connections is too large, system resources are wasted; the system resources should be wasted by keeping a certain number of connections in the connection pool available, but not too many idle connections.
At present, the following problems mainly exist in connection pool parameter configuration and management: when a system runs, the quantity of connection requests often fluctuates, and how to configure parameters of a connection pool to enable the connection pool to adapt to the fluctuation of concurrent access amount (when the user access amount reaches a peak, a sufficient quantity of connections are in the connection pool, and when the concurrent access amount is small, only a small number of connections are reserved in the connection pool), the optimization of configuration parameters of the connection pool is ensured, related technologies generally depend on self experience of operation and maintenance personnel to continuously adjust, and when the configuration is improper or insufficient, the connection and use of a database are influenced; in addition, most of the parameter configuration of the connection pool is static at present, the configured parameters cannot timely adjust the capacity of the database connection pool according to the change of the application requirements, and an application program needs to be restarted to enable the parameters to take effect when the parameters are adjusted, so that the application efficiency of database connection is influenced.
Disclosure of Invention
The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.
The embodiment of the invention provides a parameter configuration method, a parameter configuration device and a computer storage medium, which can carry out configuration adjustment on a connection pool according to application requirements.
The embodiment of the invention provides a parameter configuration method, which comprises the following steps:
acquiring start-stop time information of each connection within a preset time length; the preset time length is determined according to the analysis of the connection requirement of the application on the database;
according to the obtained start-stop time information, carrying out parameter configuration of the connection pool;
wherein the performing of the parameter configuration of the connection pool comprises:
determining concurrent connection information of each connection according to the acquired start-stop time information; wherein, the concurrent connection information includes the following part or all information: the number of concurrent connections, the concurrent connection time, and the concurrent request interval;
performing statistical analysis on the determined concurrent connection information of each connection to obtain configuration reference information;
performing parameter configuration of a connection pool according to the obtained configuration reference information;
wherein the number of concurrent connections is obtained by: taking the starting time of the current connection as the statistical starting time, and taking the starting time to the ending time of the current connection as the survival time of the current connection; counting the number of continuous connections with intersections of the survival time of the current connection from the counting starting time, and taking the number as the number of the concurrent connections of the current connection;
the concurrent connection time is obtained by the following method: after the starting times of all the continuous connections with intersections of the survival time of the current connection are sequenced according to the sequence of the starting times, the difference between the ending time of the current connection and the starting time of each connection is calculated one by one according to the sequence from the back to the front, and the first difference obtained by calculation and larger than 0 is determined as the concurrent connection time;
the concurrent request interval is obtained by: determining and calculating the starting time of the concurrent connection time corresponding to the current connection, and after the serial number sequenced by the starting time, subtracting the serial number from the number of the concurrent connections to obtain a concurrent connection correction number; and after the concurrent connection time of the current connection is divided by the concurrent connection correction number, the concurrent request interval corresponding to the current connection is obtained.
Optionally, the obtaining start-stop time information of each connection within the preset time includes:
recording start-stop time information of each connection within a preset time length through a pre-established log file;
analyzing and obtaining the start-stop time information from the recorded log file;
wherein the start-stop time information includes: the start time and the end time of the connection.
Optionally, the performing parameter configuration of the connection pool includes:
and the parameter configuration of the connection pool is carried out by calling an Application Programming Interface (API) interface.
Optionally, the configuration reference information includes part or all of the following information: a maximum value of the number of concurrent connections, a next largest value of the number of concurrent connections, a minimum value of the number of concurrent connections, a maximum value of the concurrent connection time, a maximum value of the concurrent request interval, and a minimum value of the concurrent request interval.
Optionally, the configuring, according to the obtained configuration reference information, parameters of the connection pool includes: configuring the following partial or all parameters according to the configuration reference information:
determining and configuring the following partial or all parameters according to a preset strategy according to the maximum value of the number of concurrent connections, the second largest value of the number of concurrent connections, the minimum value of the number of concurrent connections and the maximum value of the concurrent connection time:
maximum active connection number maxActive, minimum idle connection number minIdle, maximum idle connection number maxIdle maximum waiting duration maxWait;
determining the value of the longest time for which the connection is kept idle without being evicted according to the maximum value of the concurrent request interval and the minimum value of the concurrent request interval;
and configuring the minimum survival time of the connection in the connection pool according to the product of the minimum value of the concurrent request interval and the minimum value of the number of concurrent connections.
On the other hand, an embodiment of the present invention further provides a device for parameter configuration, including: an acquisition unit and a configuration unit; wherein the content of the first and second substances,
the acquisition unit is used for: acquiring start-stop time information of each connection within a preset time length; the preset time length is determined according to the analysis of the connection requirement of the application on the database;
the configuration unit is used for: and according to the acquired start-stop time information, performing parameter configuration of the connection pool.
Wherein the performing of the parameter configuration of the connection pool comprises:
determining concurrent connection information of each connection according to the acquired start-stop time information; the concurrent connection information at least comprises a concurrent connection number, concurrent connection time and/or a concurrent request interval;
performing statistical analysis on the determined concurrent connection information of each connection to obtain configuration reference information;
performing parameter configuration of a connection pool according to the obtained configuration reference information;
wherein the number of concurrent connections is obtained by: taking the starting time of the current connection as the statistical starting time, and taking the starting time to the ending time of the current connection as the survival time of the current connection; counting the number of continuous connections with intersections of the survival time of the current connection from the counting starting time, and taking the number as the number of the concurrent connections of the current connection;
the concurrent connection time is obtained by the following method: after the starting times of all the continuous connections with intersections of the survival time of the current connection are sequenced according to the sequence of the starting times, the difference between the ending time of the current connection and the starting time of each connection is calculated one by one according to the sequence from the back to the front, and the first difference obtained by calculation and larger than 0 is determined as the concurrent connection time;
the concurrent request interval is obtained by: determining and calculating the starting time of the concurrent connection time corresponding to the current connection, and after the serial number sequenced by the starting time, subtracting the serial number from the number of the concurrent connections to obtain a concurrent connection correction number; and after the concurrent connection time of the current connection is divided by the concurrent connection correction number, the concurrent request interval corresponding to the current connection is obtained.
Optionally, the obtaining unit is specifically configured to:
recording start-stop time information of each connection within a preset time length through a pre-established log file;
analyzing and obtaining the start-stop time information from the recorded log file;
wherein the start-stop time information includes: the start time and the end time of the connection.
Optionally, the configuration unit is specifically configured to:
and the parameter configuration of the connection pool is carried out by calling an Application Programming Interface (API) interface.
Optionally, the configuration reference information includes part or all of the following information: a maximum value of the number of concurrent connections, a next largest value of the number of concurrent connections, a minimum value of the number of concurrent connections, a maximum value of the concurrent connection time, a maximum value of the concurrent request interval, and a minimum value of the concurrent request interval.
Optionally, the configuring unit is configured to perform parameter configuration of the connection pool according to the obtained configuration reference information, and includes: configuring the following partial or all parameters according to the configuration reference information:
determining and configuring the following partial or all parameters according to a preset strategy according to the maximum value of the number of concurrent connections, the second largest value of the number of concurrent connections, the minimum value of the number of concurrent connections and the maximum value of the concurrent connection time:
maximum active connection number maxActive, minimum idle connection number minIdle, maximum idle connection number maxIdle maximum waiting duration maxWait;
determining the value of the longest time for which the connection is kept idle without being evicted according to the maximum value of the concurrent request interval and the minimum value of the concurrent request interval;
and configuring the minimum survival time of the connection in the connection pool according to the product of the minimum value of the concurrent request interval and the minimum value of the number of concurrent connections.
In still another aspect, an embodiment of the present invention further provides a computer storage medium, where computer-executable instructions are stored in the computer storage medium, and the computer-executable instructions are used to execute the above parameter configuration method.
Compared with the related art, the technical scheme of the application comprises the following steps: acquiring start-stop time information of each connection within a preset time length; and according to the acquired start-stop time information, performing parameter configuration of the connection pool. According to the embodiment of the invention, the configuration adjustment of the connection pool is realized according to the application requirement, and the application efficiency of database connection is improved; furthermore, parameter configuration of the connection pool is carried out by calling an Application Programming Interface (API) interface, so that configuration efficiency of the connection pool and working efficiency of application are improved.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the example serve to explain the principles of the invention and not to limit the invention.
FIG. 1 is a flow chart of a method for configuring parameters according to an embodiment of the present invention;
FIG. 2 is a block diagram of an apparatus for configuring parameters according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating an example connection start-stop time correlation according to the present invention;
FIG. 4 is a flow chart of an exemplary search algorithm applied in accordance with the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that the embodiments and features of the embodiments in the present application may be arbitrarily combined with each other without conflict.
The steps illustrated in the flow charts of the figures may be performed in a computer system such as a set of computer-executable instructions. Also, while a logical order is shown in the flow diagrams, in some cases, the steps shown or described may be performed in an order different than here.
Fig. 1 is a flowchart of a method for configuring parameters according to an embodiment of the present invention, as shown in fig. 1, including:
optionally, the obtaining of the start-stop time information of each connection within the preset time duration in the embodiment of the present invention includes:
recording start-stop time information of each connection within a preset time length through a pre-established log file;
analyzing and obtaining the start-stop time information from the recorded log file;
wherein the start-stop time information includes: the start time and the end time of the connection.
It should be noted that, in the embodiment of the present invention, the preset time duration may be determined by analyzing according to which application the connection pool is specifically applied to, for example, by analyzing according to a requirement of the application on database connection, and generally, if the connection of the application database is regular, the preset time duration may be set to 1 service period, for example, 1 day; when the application-to-database connection changes due to a particular period of time, the preset duration may be adjusted accordingly. The log file records the start-stop time information of each connection in the following mode: firstly, setting a corresponding unique identifier for each connection; respectively recording the starting time and the ending time of the connection corresponding to the identification according to the set unique identification; the start time of a connection refers to the time when the connection is acquired from the connection pool, and the end time of the connection refers to the time when the connection is returned to the connection pool.
In addition, the method for analyzing and obtaining the start-stop time information from the recorded log file in the embodiment of the present invention may include: analyzing and obtaining start-stop time information from the log file through the existing search algorithm in the related technology; wherein, the existing search algorithm comprises a log analysis algorithm familiar to the technicians in the field; the log file of the embodiment of the invention comprises a file with an extensible markup language (XML) format. The time information obtained by the analysis may include a time stamp of Linux type, which may be converted into a regular time through a formula existing in the related art.
And 102, configuring parameters of the connection pool according to the acquired start-stop time information.
Optionally, the configuring the parameters of the connection pool in the embodiment of the present invention includes:
and the parameter configuration of the connection pool is carried out by calling an Application Programming Interface (API) interface.
Optionally, the configuring the parameters of the connection pool in the embodiment of the present invention includes:
determining concurrent connection information of each connection according to the acquired start-stop time information;
performing statistical analysis on the determined concurrent connection information of each connection to obtain configuration reference information;
performing parameter configuration of a connection pool according to the obtained configuration reference information;
wherein, the concurrent connection information includes the following part or all information: number of concurrent connections, concurrent connection time, and concurrent request interval.
Optionally, in the embodiments of the present invention: the number of concurrent connections is obtained by: taking the starting time of the current connection as the statistical starting time, and taking the starting time to the ending time of the current connection as the survival time of the current connection; counting the number of continuous connections with intersections of the survival time of the current connection from the counting starting time, and taking the number as the number of the concurrent connections of the current connection;
the concurrent connection time is obtained by the following method: after the starting times of all the continuous connections with intersections of the survival time of the current connection are sequenced according to the sequence of the starting times, the difference between the ending time of the current connection and the starting time of each connection is calculated one by one according to the sequence from the back to the front, and the first difference obtained by calculation and larger than 0 is determined as the concurrent connection time; the following describes a calculation method of concurrent connections by way of example, and it is assumed that concurrent connections of connection 1 include: connection 1, connection 2, connection 3 … …, connection N-2, connection N-1, and connection N, that is, the number of concurrent connections is N, and before calculating the concurrent connection time in the embodiment of the present invention, the start times of the concurrent connections of connection 1 are arranged in order from back to front, that is, according to: the start time of connection N, the start time of connection N-1, the start time of connection N-2 … … the start time of connection 3, the start time of connection 3 are sequenced; if the difference between the termination time of the connection 1 and the start time of the connection N is greater than 0, obtaining concurrent connection time, and the subsequent concurrent connection correction number to be calculated is N; if the difference between the termination time of the connection 1 and the start time of the connection N is smaller than 0, calculating the difference between the termination time of the connection 1 and the start time of the connection N-1 according to the sequence, and if the difference is larger than 0, obtaining concurrent connection time, wherein the subsequent concurrent connection correction number required to be calculated is N-1; and if the difference between the termination time of the connection 1 and the start time of the connection N-1 is smaller than 0, calculating the difference between the termination time of the connection 1 and the start time of the connection N-2 according to the sequence, and if the difference is larger than 0, obtaining the concurrent connection time, wherein the subsequent concurrent connection correction number required to be calculated is N-2, and so on, and obtaining the concurrent connection time and the concurrent connection correction number when the difference between the termination time of the connection 1 and the start time of the connection X is larger than 0 by calculating one by one in sequence.
The concurrent request interval is obtained by: determining and calculating the starting time of the concurrent connection time corresponding to the current connection, and after the serial number sequenced by the starting time, subtracting the serial number from the number of the concurrent connections to obtain a concurrent connection correction number; and after the concurrent connection time of the current connection is divided by the concurrent connection correction number, the concurrent request interval corresponding to the current connection is obtained.
Optionally, the configuration reference information in the embodiment of the present invention includes part or all of the following information: a maximum value of the number of concurrent connections, a next largest value of the number of concurrent connections, a minimum value of the number of concurrent connections, a maximum value of the concurrent connection time, a maximum value of the concurrent request interval, and a minimum value of the concurrent request interval.
Optionally, the performing, according to the obtained configuration reference information, parameter configuration of the connection pool in the embodiment of the present invention includes: configuring the following partial or all parameters according to the configuration reference information:
determining and configuring the following partial or all parameters according to a preset strategy according to the maximum value of the number of concurrent connections, the second largest value of the number of concurrent connections, the minimum value of the number of concurrent connections and the maximum value of the concurrent connection time:
maximum active connection number maxActive, minimum idle connection number minIdle, maximum idle connection number maxIdle maximum waiting duration maxWait;
determining the value of the maximum time (minEvictableAdleTimeMillis) for which the connection is kept idle without being evicted according to the maximum value of the concurrent request interval and the minimum value of the concurrent request interval;
the time for minimum survival of a connection in the connection pool (timeBetweenEnickionRunsMillis) is configured according to the product of the minimum of the concurrent request interval and the minimum of the number of concurrent connections. the unit of the timeBetwenenvent RunsMillis millisecond, namely, each time the timeBetwenenvent RunsMillis timed, a thread is started, and a connection object with the idle time exceeding the longest time that the connection is kept idle and is not expelled in the connection pool is checked.
Compared with the related art, the technical scheme of the application comprises the following steps: acquiring start-stop time information of each connection within a preset time length; and according to the acquired start-stop time information, performing parameter configuration of the connection pool. According to the embodiment of the invention, the configuration adjustment of the connection pool is realized according to the application requirement, and the application efficiency of database connection is improved; furthermore, parameter configuration of the connection pool is carried out by calling an Application Programming Interface (API) interface, so that configuration efficiency of the connection pool and working efficiency of application are improved.
Fig. 2 is a block diagram of a device for configuring parameters according to an embodiment of the present invention, as shown in fig. 2, including: an acquisition unit and a configuration unit; wherein the content of the first and second substances,
the acquisition unit is used for: acquiring start-stop time information of each connection within a preset time length;
optionally, the obtaining unit in the embodiment of the present invention is specifically configured to:
recording start-stop time information of each connection within a preset time length through a pre-established log file;
analyzing and obtaining the start-stop time information from the recorded log file;
wherein the start-stop time information includes: the start time and the end time of the connection.
It should be noted that, in the embodiment of the present invention, the preset time duration may be determined by analyzing according to which application the connection pool is specifically applied to, for example, by analyzing according to a requirement of the application on database connection, and generally, if the connection of the application database is regular, the preset time duration may be set to 1 service period, for example, 1 day; when the application-to-database connection changes due to a particular period of time, the preset duration may be adjusted accordingly. The log file records the start-stop time information of each connection in the following mode: firstly, setting a corresponding unique identifier for each connection; respectively recording the starting time and the ending time of the connection corresponding to the identification according to the set unique identification; the start time of a connection refers to the time when the connection is acquired from the connection pool, and the end time of the connection refers to the time when the connection is returned to the connection pool.
In addition, the method for analyzing and obtaining the start-stop time information from the recorded log file in the embodiment of the present invention may include: analyzing and obtaining start-stop time information from the log file through the existing search algorithm in the related technology; wherein, the existing search algorithm comprises a log analysis algorithm familiar to the technicians in the field; the log file of the embodiment of the invention comprises a file with an extensible markup language (XML) format. The time information obtained by the analysis may include a time stamp of Linux type, which may be converted into a regular time through a formula existing in the related art.
The configuration unit is used for: and according to the acquired start-stop time information, performing parameter configuration of the connection pool.
Optionally, the configuration unit in the embodiment of the present invention is specifically configured to:
and the parameter configuration of the connection pool is carried out by calling an Application Programming Interface (API) interface.
Optionally, the configuration unit in the embodiment of the present invention is specifically configured to:
determining concurrent connection information of each connection according to the acquired start-stop time information;
performing statistical analysis on the determined concurrent connection information of each connection to obtain configuration reference information;
performing parameter configuration of a connection pool according to the obtained configuration reference information;
wherein, the concurrent connection information includes the following part or all information: number of concurrent connections, concurrent connection time, and concurrent request interval.
Optionally, the concurrent connection number according to the embodiment of the present invention is obtained by the following method: taking the starting time of the current connection as the statistical starting time, and taking the starting time to the ending time of the current connection as the survival time of the current connection; counting the number of continuous connections with intersections of the survival time of the current connection from the counting starting time, and taking the number as the number of the concurrent connections of the current connection;
the concurrent connection time is obtained by the following method: after the starting times of all the continuous connections with intersections of the survival time of the current connection are sequenced according to the sequence of the starting times, the difference between the ending time of the current connection and the starting time of each connection is calculated one by one according to the sequence from the back to the front, and the first difference obtained by calculation and larger than 0 is determined as the concurrent connection time;
the concurrent request interval is obtained by: determining and calculating the starting time of the concurrent connection time corresponding to the current connection, and after the serial number sequenced by the starting time, subtracting the serial number from the number of the concurrent connections to obtain a concurrent connection correction number; and after the concurrent connection time of the current connection is divided by the concurrent connection correction number, the concurrent request interval corresponding to the current connection is obtained.
Optionally, the configuration reference information in the embodiment of the present invention includes part or all of the following information: a maximum value of the number of concurrent connections, a next largest value of the number of concurrent connections, a minimum value of the number of concurrent connections, a maximum value of the concurrent connection time, a maximum value of the concurrent request interval, and a minimum value of the concurrent request interval.
Optionally, the configuring unit in the embodiment of the present invention is configured to perform parameter configuration on the connection pool according to the obtained configuration reference information, and includes: configuring the following partial or all parameters according to the configuration reference information:
determining and configuring the following partial or all parameters according to a preset strategy according to the maximum value of the number of concurrent connections, the second largest value of the number of concurrent connections, the minimum value of the number of concurrent connections and the maximum value of the concurrent connection time:
maximum active connection number maxActive, minimum idle connection number minIdle, maximum idle connection number maxIdle maximum waiting duration maxWait;
determining the value of the longest time for which the connection is kept idle without being evicted according to the maximum value of the concurrent request interval and the minimum value of the concurrent request interval;
and configuring the minimum survival time of the connection in the connection pool according to the product of the minimum value of the concurrent request interval and the minimum value of the number of concurrent connections.
Compared with the related art, the technical scheme of the application comprises the following steps: acquiring start-stop time information of each connection within a preset time length; and according to the acquired start-stop time information, performing parameter configuration of the connection pool. According to the embodiment of the invention, the configuration adjustment of the connection pool is realized according to the application requirement, and the application efficiency of database connection is improved; furthermore, parameter configuration of the connection pool is carried out by calling an Application Programming Interface (API) interface, so that configuration efficiency of the connection pool and working efficiency of application are improved.
The embodiment of the invention also provides a computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are used for executing the parameter configuration method.
The method of the embodiment of the present invention is described in detail below by using application examples, which are only used for illustrating the present invention and are not used for limiting the protection scope of the present invention.
Application example
The application example of the invention carries out the configuration of the parameters of the connection pool according to the running log, and dynamically adjusts the configuration parameters through the background management interface, thereby obtaining the parameter value which is most suitable for the current application condition, enabling the configuration to take effect under the condition of not restarting the Web server, and avoiding the influence on the performance of the connection pool when the parameters are unreasonably configured. The main architecture of the application example of the invention comprises: establishing a log file in an XML format, and recording the time for acquiring the connection from the connection pool and the time for returning the connection to the connection pool; and then analyzing the running log through a search algorithm to obtain a group of concurrent connection number, concurrent time and an interval of concurrent requests, and taking the group of concurrent connection number, concurrent time and interval of concurrent requests as a reference of parameter values. And then, parameters are updated on line by using a background management interface, so that the performance of the application is improved.
Based on the connection is sequentially allocated, when the start-stop time information of the connection is recorded by a log file, the application example of the invention sets a corresponding code (RecordID) for each connection, and the start time of the connection coded later is certain to be greater than that of the connection coded earlier; FIG. 3 is a diagram illustrating the connection start-stop time relationship according to an exemplary application of the present invention, wherein the start time of a connection encoded later is greater than the start time of a connection encoded earlier by a greater value;
the application example of the invention adopts the existing search algorithm in the related technology to analyze the log files, and the basic thought is as follows: and searching the subsequent connections by taking the survival time of each connection as a reference, and if the starting time of a certain subsequent connection is less than the ending time of the reference connection, adding 1 to the counter to indicate that the connection is the concurrent connection of the current connection. If the end time of the reference connection is after the end time of a certain subsequent connection, the end time for calculating the concurrency time is set as the end time of the connection. The concurrency request interval can be obtained according to the concurrency number of each group and the time difference of the head and the tail two concurrent connections of the group. FIG. 4 is a flowchart of an exemplary search algorithm applied to the present invention, as shown in FIG. 4, including:
It should be noted that, the sequence number of the start time sequence implemented by the present invention is an arithmetic sequence with 1 as the start number and 1 as the difference, that is, the sequence number of the start time sequence is: 1. 2, 3, 4, 5 … ….
The number of concurrent connections, the concurrent connection time, and the concurrent request interval obtained by the above algorithm are all a set of values. Analyzing the running logs in different time periods to obtain enough analysis samples, determining the maximum value, the second largest value, the minimum value and the maximum concurrent connection time of the concurrent connection number by using the existing statistical method in the related technology, such as statistics according to frequency, and taking the maximum value, the second largest value, the minimum value and the maximum concurrent connection time as the value reference of maxActive, maxIdle, minIdle and maxWait parameters; finding out the minimum concurrent request interval and the maximum concurrent request interval, and taking the minimum concurrent request interval and the maximum concurrent request interval as the value reference of the longest time parameter of connection keeping idle without being evicted; the product of the minimum concurrent request interval and the minimum concurrent connection number can be used as a value reference of a minimum survival time parameter connected in the connection pool;
the application example of the invention can obtain the configuration parameters of the connection pool which best meets the requirements of the current application by periodically analyzing the operation log. The first startup of the connection pool is operated according to the parameters configured by the user, and the last log operation result is loaded in each subsequent startup to dynamically determine the value of the pool parameters, so that the parameter configuration is effectively carried out according to the result.
Aiming at the problems that configuration strategies of a connection pool are mostly static and a Web server needs to be restarted when parameters become effective, the application example of the invention automatically reads the analysis data from an XML log file and then calls an API (application program interface) of the connection pool of the database to configure new parameter values, so that new configuration becomes effective under the condition of not interrupting Web service and the use is flexible. The application example of the invention reduces the response time of the system and the cost of system resources, thereby improving the overall performance of the system.
It will be understood by those skilled in the art that all or part of the steps of the above methods may be implemented by a program instructing associated hardware (e.g., a processor) to perform the steps, and the program may be stored in a computer readable storage medium, such as a read only memory, a magnetic or optical disk, and the like. Alternatively, all or part of the steps of the above embodiments may be implemented using one or more integrated circuits. Accordingly, each module/unit in the above embodiments may be implemented in hardware, for example, by an integrated circuit to implement its corresponding function, or in software, for example, by a processor executing a program/instruction stored in a memory to implement its corresponding function. The present invention is not limited to any specific form of combination of hardware and software.
Although the embodiments of the present invention have been described above, the above description is only for the convenience of understanding the present invention, and is not intended to limit the present invention. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (11)
1. A method of parameter configuration, comprising:
acquiring start-stop time information of each connection within a preset time length; the preset time length is determined according to the analysis of the connection requirement of the application on the database;
according to the obtained start-stop time information, carrying out parameter configuration of the connection pool;
wherein the performing of the parameter configuration of the connection pool comprises:
determining concurrent connection information of each connection according to the acquired start-stop time information; wherein, the concurrent connection information includes the following part or all information: the number of concurrent connections, the concurrent connection time, and the concurrent request interval;
performing statistical analysis on the determined concurrent connection information of each connection to obtain configuration reference information;
performing parameter configuration of a connection pool according to the obtained configuration reference information;
wherein the number of concurrent connections is obtained by: taking the starting time of the current connection as the statistical starting time, and taking the starting time to the ending time of the current connection as the survival time of the current connection; counting the number of continuous connections with intersections of the survival time of the current connection from the counting starting time, and taking the number as the number of the concurrent connections of the current connection;
the concurrent connection time is obtained by the following method: after the starting times of all the continuous connections with intersections of the survival time of the current connection are sequenced according to the sequence of the starting times, the difference between the ending time of the current connection and the starting time of each connection is calculated one by one according to the sequence from the back to the front, and the first difference obtained by calculation and larger than 0 is determined as the concurrent connection time;
the concurrent request interval is obtained by: determining and calculating the starting time of the concurrent connection time corresponding to the current connection, and after the serial number sequenced by the starting time, subtracting the serial number from the number of the concurrent connections to obtain a concurrent connection correction number; and after the concurrent connection time of the current connection is divided by the concurrent connection correction number, the concurrent request interval corresponding to the current connection is obtained.
2. The method according to claim 1, wherein the obtaining start-stop time information of each connection within a preset time duration comprises:
recording start-stop time information of each connection within a preset time length through a pre-established log file;
analyzing and obtaining the start-stop time information from the recorded log file;
wherein the start-stop time information includes: the start time and the end time of the connection.
3. The method of claim 1, wherein the performing parameter configuration of the connection pool comprises:
and the parameter configuration of the connection pool is carried out by calling an Application Programming Interface (API) interface.
4. The method according to any one of claims 1 to 3, wherein the configuration reference information comprises part or all of the following information: a maximum value of the number of concurrent connections, a next largest value of the number of concurrent connections, a minimum value of the number of concurrent connections, a maximum value of the concurrent connection time, a maximum value of the concurrent request interval, and a minimum value of the concurrent request interval.
5. The method according to claim 4, wherein the configuring parameters of the connection pool according to the obtained configuration reference information comprises: configuring the following partial or all parameters according to the configuration reference information:
determining and configuring the following partial or all parameters according to a preset strategy according to the maximum value of the number of concurrent connections, the second largest value of the number of concurrent connections, the minimum value of the number of concurrent connections and the maximum value of the concurrent connection time:
maximum active connection number maxActive, minimum idle connection number minIdle, maximum idle connection number maxIdle maximum waiting duration maxWait;
determining the value of the longest time for which the connection is kept idle without being evicted according to the maximum value of the concurrent request interval and the minimum value of the concurrent request interval;
configuring the time of minimum survival of the connection in the connection pool according to the product of the minimum value of the concurrent request interval and the minimum value of the number of concurrent connections.
6. An apparatus for parameter configuration, comprising: an acquisition unit and a configuration unit; wherein the content of the first and second substances,
the acquisition unit is used for: acquiring start-stop time information of each connection within a preset time length; the preset time length is determined according to the analysis of the connection requirement of the application on the database;
the configuration unit is used for: according to the obtained start-stop time information, carrying out parameter configuration of the connection pool;
wherein the performing of the parameter configuration of the connection pool comprises:
determining concurrent connection information of each connection according to the acquired start-stop time information; the concurrent connection information at least comprises a concurrent connection number, concurrent connection time and/or a concurrent request interval;
performing statistical analysis on the determined concurrent connection information of each connection to obtain configuration reference information;
performing parameter configuration of a connection pool according to the obtained configuration reference information;
wherein the number of concurrent connections is obtained by: taking the starting time of the current connection as the statistical starting time, and taking the starting time to the ending time of the current connection as the survival time of the current connection; counting the number of continuous connections with intersections of the survival time of the current connection from the counting starting time, and taking the number as the number of the concurrent connections of the current connection;
the concurrent connection time is obtained by the following method: after the starting times of all the continuous connections with intersections of the survival time of the current connection are sequenced according to the sequence of the starting times, the difference between the ending time of the current connection and the starting time of each connection is calculated one by one according to the sequence from the back to the front, and the first difference obtained by calculation and larger than 0 is determined as the concurrent connection time;
the concurrent request interval is obtained by: determining and calculating the starting time of the concurrent connection time corresponding to the current connection, and after the serial number sequenced by the starting time, subtracting the serial number from the number of the concurrent connections to obtain a concurrent connection correction number; and after the concurrent connection time of the current connection is divided by the concurrent connection correction number, the concurrent request interval corresponding to the current connection is obtained.
7. The apparatus according to claim 6, wherein the obtaining unit is specifically configured to:
recording start-stop time information of each connection within a preset time length through a pre-established log file;
analyzing and obtaining the start-stop time information from the recorded log file;
wherein the start-stop time information includes: the start time and the end time of the connection.
8. The apparatus according to claim 6, wherein the configuration unit is specifically configured to:
and the parameter configuration of the connection pool is carried out by calling an Application Programming Interface (API) interface.
9. The apparatus according to any one of claims 6 to 8, wherein the configuration reference information comprises part or all of the following information: a maximum value of the number of concurrent connections, a next largest value of the number of concurrent connections, a minimum value of the number of concurrent connections, a maximum value of the concurrent connection time, a maximum value of the concurrent request interval, and a minimum value of the concurrent request interval.
10. The apparatus according to claim 9, wherein the configuring unit is configured to perform parameter configuration of a connection pool according to the obtained configuration reference information, and includes: configuring the following partial or all parameters according to the configuration reference information:
determining and configuring the following partial or all parameters according to a preset strategy according to the maximum value of the number of concurrent connections, the second largest value of the number of concurrent connections, the minimum value of the number of concurrent connections and the maximum value of the concurrent connection time:
maximum active connection number maxActive, minimum idle connection number minIdle, maximum idle connection number maxIdle maximum waiting duration maxWait;
determining the value of the longest time for which the connection is kept idle without being evicted according to the maximum value of the concurrent request interval and the minimum value of the concurrent request interval;
and configuring the minimum survival time of the connection in the connection pool according to the product of the minimum value of the concurrent request interval and the minimum value of the number of concurrent connections.
11. A computer storage medium having computer-executable instructions stored therein for performing the method of parameter configuration of any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810864294.0A CN108920722B (en) | 2018-08-01 | 2018-08-01 | Parameter configuration method and device and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810864294.0A CN108920722B (en) | 2018-08-01 | 2018-08-01 | Parameter configuration method and device and computer storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108920722A CN108920722A (en) | 2018-11-30 |
| CN108920722B true CN108920722B (en) | 2021-03-19 |
Family
ID=64394218
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810864294.0A Active CN108920722B (en) | 2018-08-01 | 2018-08-01 | Parameter configuration method and device and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108920722B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115237429A (en) * | 2022-07-18 | 2022-10-25 | 江苏卓易信息科技股份有限公司 | Cloud server test verification method based on firmware dynamic parameter adjustment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102024008A (en) * | 2009-09-09 | 2011-04-20 | 浙江省电力公司 | Limitation method of Web server concurrent number |
| CN104239365A (en) * | 2013-06-24 | 2014-12-24 | 北京新媒传信科技有限公司 | Database connection pool and method for acquiring links in database connection pool |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9569472B2 (en) * | 2013-06-06 | 2017-02-14 | Oracle International Corporation | System and method for providing a second level connection cache for use with a database environment |
| CN104915412B (en) * | 2015-06-05 | 2018-07-03 | 北京京东尚科信息技术有限公司 | A kind of method and system of dynamic management data library connection |
| CN105490850B (en) * | 2015-12-09 | 2018-10-19 | 北京京东尚科信息技术有限公司 | The method and system of disposition data source |
| CN106560785A (en) * | 2016-06-01 | 2017-04-12 | 北京乾丰益信息技术有限公司 | Application system management platform and data processing method thereof |
| CN106250500A (en) * | 2016-07-29 | 2016-12-21 | 广州唯品会信息科技有限公司 | The dynamic management approach of database connection pool and system |
| CN106302794B (en) * | 2016-08-31 | 2019-12-03 | 东软集团股份有限公司 | The dynamic setting method and device of Connecting quantity |
-
2018
- 2018-08-01 CN CN201810864294.0A patent/CN108920722B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102024008A (en) * | 2009-09-09 | 2011-04-20 | 浙江省电力公司 | Limitation method of Web server concurrent number |
| CN104239365A (en) * | 2013-06-24 | 2014-12-24 | 北京新媒传信科技有限公司 | Database connection pool and method for acquiring links in database connection pool |
Non-Patent Citations (2)
| Title |
|---|
| 一种提高连接池并发处理性能的设计方案;马海燕;《微计算机信息》;20060614;第167-169,294页 * |
| 基于负荷预测的数据库连接池设计与实现;李运莉;《计算机与数字工程》;20051024;第144-146页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108920722A (en) | 2018-11-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11379272B2 (en) | Autoscaling using file access or cache usage for cluster machines | |
| Cho et al. | Natjam: Design and evaluation of eviction policies for supporting priorities and deadlines in mapreduce clusters | |
| CN112346829A (en) | Method and equipment for task scheduling | |
| KR20200122364A (en) | Resource scheduling method and terminal device | |
| WO2021093365A1 (en) | Gpu video memory management control method and related device | |
| CN110109741B (en) | Method and device for managing circular tasks, electronic equipment and storage medium | |
| CN109582649B (en) | Metadata storage method, device and equipment and readable storage medium | |
| CN109800269A (en) | Data managing method, device, computer equipment and storage medium | |
| CN110807145A (en) | Query engine acquisition method, device and computer-readable storage medium | |
| CN110297743B (en) | Load testing method and device and storage medium | |
| CN111061690B (en) | RAC-based database log file reading method and device | |
| CN108475201A (en) | A kind of data capture method in virtual machine start-up course and cloud computing system | |
| CN108920722B (en) | Parameter configuration method and device and computer storage medium | |
| CN112860387A (en) | Distributed task scheduling method and device, computer equipment and storage medium | |
| CN113886426A (en) | Data query method and device, computer equipment and storage medium | |
| CN116089477B (en) | Distributed training method and system | |
| CN110650206A (en) | IO flow control method and device in distributed storage system and storage medium | |
| US20230100110A1 (en) | Computing resource management method, electronic equipment and program product | |
| CN115422010A (en) | Node management method and device in data cluster and storage medium | |
| CN112541101B (en) | Subscription data pushing method and device, electronic equipment and computer storage medium | |
| CN109656708B (en) | Animation playing limiting method for Android, storage medium, electronic equipment and system | |
| CN113127179A (en) | Resource scheduling method and device, electronic equipment and computer readable medium | |
| CN111563106A (en) | Data caching method, device and system and readable storage medium | |
| CN110365342A (en) | Waveform decoder method and device | |
| US11755534B2 (en) | Data caching method and node based on hyper-converged infrastructure |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210208 Address after: Building 9, No.1, guanpu Road, Guoxiang street, Wuzhong Economic Development Zone, Wuzhong District, Suzhou City, Jiangsu Province Applicant after: SUZHOU LANGCHAO INTELLIGENT TECHNOLOGY Co.,Ltd. Address before: Room 1601, floor 16, 278 Xinyi Road, Zhengdong New District, Zhengzhou City, Henan Province Applicant before: ZHENGZHOU YUNHAI INFORMATION TECHNOLOGY Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |