CN116166513A - Evaluation method, device and storage medium for database performance test - Google Patents
Evaluation method, device and storage medium for database performance test Download PDFInfo
- Publication number
- CN116166513A CN116166513A CN202310078383.3A CN202310078383A CN116166513A CN 116166513 A CN116166513 A CN 116166513A CN 202310078383 A CN202310078383 A CN 202310078383A CN 116166513 A CN116166513 A CN 116166513A
- Authority
- CN
- China
- Prior art keywords
- performance
- index
- tested
- data
- weight vector
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000011156 evaluation Methods 0.000 title claims abstract description 92
- 238000011056 performance test Methods 0.000 title claims abstract description 50
- 238000000034 method Methods 0.000 claims abstract description 54
- 238000012360 testing method Methods 0.000 claims abstract description 45
- 239000011159 matrix material Substances 0.000 claims description 88
- 238000010276 construction Methods 0.000 claims description 33
- 238000004422 calculation algorithm Methods 0.000 claims description 27
- 238000004364 calculation method Methods 0.000 claims description 19
- 230000004044 response Effects 0.000 claims description 11
- 230000005484 gravity Effects 0.000 claims description 8
- 230000009466 transformation Effects 0.000 claims description 5
- 230000008569 process Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 8
- 238000004590 computer program Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 238000012545 processing Methods 0.000 description 6
- 230000009286 beneficial effect Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000009897 systematic effect Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 238000013210 evaluation model Methods 0.000 description 1
- 238000012854 evaluation process Methods 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000009662 stress testing Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3409—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3447—Performance evaluation by modeling
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The application discloses an evaluation method, equipment and storage medium for database performance test. The method comprises the following steps: determining an evaluation object set corresponding to a database to be tested, and determining a performance index system to be tested and a data index system to be tested based on test requirements; determining a performance index weight vector corresponding to a performance index system to be tested; determining a subjective data index weight vector and an objective data index weight vector corresponding to a data index system to be tested; determining a data index weight vector corresponding to the data index system to be tested based on the subjective data index weight vector and the objective data index weight vector; based on the data index weight vector, determining a performance evaluation index corresponding to each performance index in the performance index system to be tested, and based on the performance evaluation index and the performance index weight vector, determining a comprehensive evaluation index of the database to be tested. The method realizes a set of performance test evaluation schemes capable of selecting a proper database for the application system.
Description
Technical Field
The present disclosure relates to the field of database testing technologies, and in particular, to a method, an apparatus, and a storage medium for evaluating database performance testing.
Background
The database performance comparison test is to simulate various normal, peak and abnormal load conditions by an automatic test tool to test various performance indexes of the database. The main types of performance testing include load testing and stress testing. Load testing is the testing of changes in various performance metrics of a system as the load increases gradually in order to determine the performance of the system under various workloads. The pressure test is to test the change condition of various performance indexes of the system under the condition of high load and the hidden functional trouble and stability under the load peak value.
For an application system, the performance of the application system is directly affected by the performance of the database, and the performance effects brought by different databases are different. Through the database performance comparison test, a proper database can be selected for the application system, so that positive guidance is provided for perfecting and improving the performance of the application system, and meanwhile, reliable guarantee is provided for the quality of the application system.
And selecting a proper database for the application system, and testing the databases to be compared by applying a comprehensive and comprehensive database performance test scheme based on the measured data under the same specific condition. However, there is currently no set of suitable database performance test evaluation schemes that can select a suitable database for an application system.
Disclosure of Invention
The embodiment of the application provides an evaluation method, equipment and a storage medium for database performance test, which solve the technical problem that no set of suitable evaluation scheme for database performance test exists at present and a suitable database can be selected for an application system.
In a first aspect, an embodiment of the present application provides an evaluation method for a database performance test, where the method includes: determining an evaluation object set corresponding to a database to be tested, and determining a performance index system to be tested and a data index system to be tested based on test requirements; the performance index system to be tested comprises a performance index for evaluating the database to be tested; the data index system to be tested comprises data indexes for evaluating single performance; the performance indexes comprise: query performance, delete performance, update performance, insert performance; the data index includes: average response time, average throughput, average CPU utilization, average memory occupancy, and anomaly rate; based on a preset performance index correlation judgment matrix, determining a performance index weight vector corresponding to a performance index system to be tested through a preset performance weight vector construction algorithm; based on a preset data index correlation judgment matrix, determining a subjective data index weight vector corresponding to a data index system to be tested through a preset data weight vector construction algorithm, and determining an objective data index weight vector corresponding to the data index system to be tested based on measured data obtained by performing performance test on a database to be tested; determining a data index weight vector corresponding to the data index system to be tested based on the subjective data index weight vector and the objective data index weight vector; based on the data index weight vector, determining a performance evaluation index corresponding to each performance index in the performance index system to be tested, and based on the performance evaluation index and the performance index weight vector, determining a comprehensive evaluation index of the database to be tested.
In one implementation manner of the present application, before determining, based on a preset performance index correlation judgment matrix and through a preset performance weight vector construction algorithm, a performance index weight vector corresponding to a performance index system to be tested, the method further includes: determining the relative importance value of each performance index relative to other performance indexes based on a preset judgment matrix element value table; based on the relative importance value, a performance index correlation judgment matrix is constructed and is expressed by the following formula:
wherein lambda is a performance index correlation judgment matrix lambda i,j Representing performance index I i For performance index I j Is of relative importance, lambda i,j λ j,i ＝1(,j＝1,2,3,4)。
In one implementation of the present application, the performance weight vector construction algorithm is represented by the following formula:
wherein W is a performance index weight vector, alpha is a construction vector, and alpha is expressed by the following formula:
in one implementation manner of the present application, after determining the performance index weight vector corresponding to the performance index system to be tested, the method further includes: the rationality evaluation is carried out on the performance index weight vector, and the method specifically comprises the following steps: calculating the consistency value of the performance index correlation judgment matrix and the performance index weight vector through a preset consistency index calculation formula; based on the consistency value, the consistency ratio of the performance index correlation judgment matrix and the performance index weight vector is determined, and the performance index weight vector is determined to pass through rationality evaluation under the condition that the consistency ratio is smaller than a preset threshold value.
In one implementation of the present application, the consistency index calculation formula is represented by the following formula:
wherein CI is a consistency index, W is a performance index weight vector, Λ is a performance index correlation judgment matrix, and W i The ith performance index weight coefficient in the performance index weight vector is used as the ith performance index weight coefficient;
the consistency ratio is calculated from the following formula:
wherein CR is a consistency ratio, and CI is a consistency index.
In one implementation manner of the present application, determining an objective data index weight vector corresponding to a data index system to be tested based on measured data obtained by performing a performance test on a database to be tested specifically includes: based on measured data obtained by performance test of a database to be tested, constructing a measured data matrix; performing standardized transformation on the measured data matrix to obtain a corresponding standardized matrix, and determining a corresponding index specific gravity matrix based on the standardized matrix; determining the information entropy of the data index through a preset data index information entropy calculation formula based on the index proportion matrix; based on the information entropy of the data index, determining an objective data index weight vector corresponding to the data index system to be tested through a preset objective data index weight vector calculation formula.
In one implementation of the present application, the normalized transformation of the measured data matrix is implemented by the following formula:
wherein ,Θk In order to normalize the matrix,
representing database object D to be tested i In terms of performance index I k Data index corresponding to the execution time->
Is a measured data of (1);
performance index I k The corresponding measured data matrix is represented by the following formula:
wherein ,Ξk Representing the measured data matrix;
the index specific gravity matrix is determined by the following formula:
wherein ,Ψk Is an index specific gravity matrix;
the data index information entropy calculation formula is represented by the following formula:
wherein ,
entropy of data index, if there is +.>
Let ∈>
In one implementation of the present application, the performance evaluation index corresponding to the performance index is determined by the following formula:
wherein ,p(Ik ,D i ) For the database D to be tested i Performance index I of (2) k A corresponding performance evaluation index;
is the performance index I k Corresponding data index weight vector W k The j-th data index weight coefficient of (a);
the comprehensive evaluation index of the database to be tested is determined by the following formula:
wherein ,p(Di ) Represents the comprehensive evaluation index, w, corresponding to the ith database to be tested j And the j-th performance index weight coefficient in the performance index weight vector is used as the j-th performance index weight coefficient.
In a second aspect, an embodiment of the present application further provides an evaluation device for a database performance test, where the device includes: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executable by the at least one processor to enable the at least one processor to: determining an evaluation object set corresponding to a database to be tested, and determining a performance index system to be tested and a data index system to be tested based on test requirements; the performance index system to be tested comprises a performance index for evaluating the database to be tested; the data index system to be tested comprises data indexes for evaluating single performance; the performance indexes comprise: query performance, delete performance, update performance, insert performance; the data index includes: average response time, average throughput, average CPU utilization, average memory occupancy, and anomaly rate; based on a preset performance index correlation judgment matrix, determining a performance index weight vector corresponding to a performance index system to be tested through a preset performance weight vector construction algorithm; based on a preset data index correlation judgment matrix, determining a subjective data index weight vector corresponding to a data index system to be tested through a preset data weight vector construction algorithm, and determining an objective data index weight vector corresponding to the data index system to be tested based on measured data obtained by performing performance test on a database to be tested; determining a data index weight vector corresponding to the data index system to be tested based on the subjective data index weight vector and the objective data index weight vector; based on the data index weight vector, determining a performance evaluation index corresponding to each performance index in the performance index system to be tested, and based on the performance evaluation index and the performance index weight vector, determining a comprehensive evaluation index of the database to be tested.
In a third aspect, embodiments of the present application further provide a non-volatile computer storage medium storing computer-executable instructions for evaluating database performance tests, wherein the computer-executable instructions are configured to: determining an evaluation object set corresponding to a database to be tested, and determining a performance index system to be tested and a data index system to be tested based on test requirements; the performance index system to be tested comprises a performance index for evaluating the database to be tested; the data index system to be tested comprises data indexes for evaluating single performance; the performance indexes comprise: query performance, delete performance, update performance, insert performance; the data index includes: average response time, average throughput, average CPU utilization, average memory occupancy, and anomaly rate; based on a preset performance index correlation judgment matrix, determining a performance index weight vector corresponding to a performance index system to be tested through a preset performance weight vector construction algorithm; based on a preset data index correlation judgment matrix, determining a subjective data index weight vector corresponding to a data index system to be tested through a preset data weight vector construction algorithm, and determining an objective data index weight vector corresponding to the data index system to be tested based on measured data obtained by performing performance test on a database to be tested; determining a data index weight vector corresponding to the data index system to be tested based on the subjective data index weight vector and the objective data index weight vector; based on the data index weight vector, determining a performance evaluation index corresponding to each performance index in the performance index system to be tested, and based on the performance evaluation index and the performance index weight vector, determining a comprehensive evaluation index of the database to be tested.
The evaluation method, the device and the storage medium for the database performance test have the following beneficial effects:
1. the method is implemented based on the Jmeter measured data and can complete the standardized processing of the measured data, fully integrate the measured data and deeply mine the information contained in the measured data, and improve the utilization rate of the data.
2. The method establishes a systematic and hierarchical multi-level index evaluation system according to the test requirements, and combines a hierarchical analysis method and an entropy weight method to determine the weight coefficient of each level of evaluation system, so that the subjective intention of a decision maker can be reflected, objective measurement data can be fully utilized, the importance degree of each index can be quantified scientifically, and the rationality and accuracy of performance comparison test results are improved.
3. The method completes quantitative scoring of the database in a plurality of performance tests by constructing a reasonable index evaluation system and combining an effective weight calculation method, realizes objective and reasonable comprehensive evaluation of the database, provides scientific reference and positive guidance for selecting a proper database for an application system, improves and promotes the performance of the application system, and ensures and increases the quality and reliability of the application system.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute an undue limitation to the application. In the drawings:
FIG. 1 is a flowchart of an evaluation method for database performance test according to an embodiment of the present application;
fig. 2 is a schematic diagram of an internal structure of an evaluation device for database performance test according to an embodiment of the present application.
Detailed Description
For the purposes, technical solutions and advantages of the present application, the technical solutions of the present application will be clearly and completely described below with reference to specific embodiments of the present application and corresponding drawings. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
Performance comparison tests are performed on different databases, requiring the aid of automated test tools. At present, there are many related test tools, jmeter is a database test tool with better performance test, and meanwhile, the Jmeter is mature in terms of realizing the call of various interfaces, and can well support various common interfaces, such as HTTP (S), webService, JDBC, JAVA, FTP, and the like. As open source software, the Jmeter also has the characteristics of strong expansibility and perfect functions, supports a plurality of plug-ins, supports performance test on different types of applications or services, supports a graphical interface and a command line mode, supports various forms to display performance test results, and can provide a graphical analysis function for the test results. The use of a Jmeter to perform performance comparison test on a database is mainly used for obtaining time characteristic data (response time) and resource characteristic data (throughput, CPU and memory use conditions) of different databases under certain specific conditions, and can provide reasonable data support for subsequent database performance comparison. In order to ensure the rigor of the related performance test, the consistency of the test environment and the test condition needs to be maintained, and the method mainly comprises an operating system, an application software version, hardware configuration, a test data scale, concurrent setting in the test process, concurrent increasing and changing processes and the like.
After performance tests are performed on different databases by using a Jmeter to obtain actual measurement data, an index evaluation system and corresponding weight coefficients need to be determined. When using several metrics to comprehensively evaluate different databases, the evaluation of the metrics on the databases is not equally important. In the comprehensive evaluation model, the magnitude of the weight coefficient reflects the importance degree of the evaluation index, and the larger the weight coefficient is, the larger the importance degree of the index is. Methods for determining index weight coefficients are generally classified into three types: subjective right-determining method, objective right-determining method and combined integrated right-determining method. The analytic hierarchy process is one subjective and quantitative combined systematic and hierarchical analysis process and features that the hierarchical structure is established to convert the judgment of person into the comparison of importance between several factors, so that the qualitative judgment difficult to quantify is converted into the comparison of operational importance and the weight coefficient of the index is further calculated to determine. The entropy weight method is an objective weight giving method and is mainly characterized in that the entropy weight of each index is calculated by utilizing information entropy through processing known data and according to the dispersion degree of the data of each index, so that objective index weight is obtained. Based on the thought of the combined integrated weight determining method, the weight determining scheme formed by combining the analytic hierarchy process and the entropy weight method can reflect subjective demand willingness of a decision maker, fully utilize actual measurement data and objectively reflect the importance degree of each index.
The embodiment of the application provides an evaluation method, equipment and a storage medium for database performance test, which solve the technical problem that no set of suitable evaluation scheme for database performance test exists at present and a suitable database can be selected for an application system.
The following describes in detail the technical solution proposed in the embodiments of the present application through the accompanying drawings.
Fig. 1 is a flowchart of an evaluation method for database performance test according to an embodiment of the present application. As shown in fig. 1, an evaluation method for database performance test provided in the embodiment of the present application specifically includes the following steps:
In one embodiment of the present application, the evaluation object set composed of n databases to be tested is d= { D 1 ，D 2 ，…，D n}, wherein Di Representing different database objects to be tested (i=1, 2, …, n).
In one embodiment of the application, the determined performance index system to be tested comprises performance indexes for evaluating the database to be tested based on the test requirements; the data index system to be tested comprises data indexes for evaluating single performance; the performance indexes comprise: query performance, delete performance, update performance, insert performance; the data index includes: average response time, average throughput, average CPU usage, average memory occupancy, and anomaly.
In one embodiment of the present application, the performance index system under test may be expressed as i= { I 1 ，I 2 ，I 3 ，I 4}, wherein ,I1 、I 2 、I 3 、I 4 Respectively represent databasesQuery performance, deletion performance, update performance, insertion performance. The performance index weight vector of the performance index system to be measured can be expressed as
W＝[w 1 ，w 2 ，w 3 ，w 4 ] T ，
wherein wj 0 or more represents the performance index I j Weight coefficient (j=1, 2,3, 4) at the time of evaluating database, and satisfies
In one embodiment of the present application, the data-under-test index hierarchy may be expressed as
wherein />
The average response time, the average throughput, the average CPU usage, the average memory occupancy and the anomaly rate are respectively represented.
It should be noted that, among the above 5 data indexes, the anomaly rate of all databases to be tested in a single performance test may be zero, which results in that the data index of anomaly rate cannot play a role in evaluating the single performance test. Discarding the data metrics in this case does not affect the final performance contrast test, and therefore, without loss of generality, it can be assumed that all databases under test have a non-zero anomaly rate in a single performance test when the scheme is described below. The performance index system to be measured is used for evaluating the performance index I k The data index weight vector of (2) may be expressed as
wherein 
Representing data index +.>
In evaluating the single item Performance test I k Weight coefficient (j=1, 2,3,4, 5) at the time and satisfies +.>
In one embodiment of the present application, after determining the performance index system to be tested, a performance index weight vector corresponding to the performance index system to be tested is determined through a preset performance weight vector construction algorithm based on a preset performance index correlation judgment matrix.
Specifically, according to the performance index system I to be tested, determining a performance index related judgment matrix by means of scoring by an expert or a tester:
wherein λi,j Representing performance index I when evaluating a database to be tested i For performance index I j The relative importance value of (2) can only be selected from a preset judgment matrix element value table. From the value characteristics of the elements in the performance index correlation judgment matrix lambda, lambda can be seen i,j λ j,i =1 (, j=1, 2,3, 4), so the entire matrix can be determined by determining only 6 elements of the triangle on the performance index correlation determination matrix Λ.
The judgment matrix element value table is as follows:
further, the construction vector α is determined based on the performance index correlation judgment matrix Λ, and α is expressed by the following formula:
further, based on the construction vector α, a performance index weight vector is determined by a performance weight vector construction algorithm, which is expressed by the following formula:
in one embodiment of the present application, since the above-mentioned evaluation process involves a plurality of indexes, in the process of determining the importance of the indexes, that is, when determining the performance index correlation determination matrix Λ, a contradiction phenomenon that the determination results are inconsistent easily occurs, and therefore, after determining the performance index weight vector W, a consistency check is required. To perform a rationality assessment of the performance index weight vector determination process.
In one embodiment of the present application, the consistency check specifically includes: calculating the consistency value of the performance index correlation judgment matrix and the performance index weight vector through a preset consistency index calculation formula; based on the consistency value, the consistency ratio of the performance index correlation judgment matrix and the performance index weight vector is determined, and under the condition that the consistency ratio is smaller than a preset threshold value, the performance index weight vector is determined to pass through rationality evaluation, otherwise, the performance index correlation judgment matrix is required to be corrected so as to redetermine the performance index weight vector.
In one implementation of the present application, the consistency index calculation formula is represented by the following formula:
wherein CI is a consistency index, W is a performance index weight vector, Λ is a performance index correlation judgment matrix, and W i For the ith performance index weight in the performance index weight vectorA weight coefficient;
the consistency ratio is calculated from the following formula:
wherein CR is a consistency ratio, and CI is a consistency index.
In one embodiment of the present application, after determining the data index system to be measured, first, based on a preset data index correlation judgment matrix, a subjective data index weight vector corresponding to the data index system to be measured is determined through a preset data weight vector construction algorithm.
Specifically, according to the index system of the data to be measured
Determining a data index correlation judgment matrix by means of scoring by an expert or a tester:
wherein ,
is expressed in evaluating performance index I k Data index->
For index->
Numerical value of relative importance of (2)The value can only be selected from the value table of the judgment matrix elements. It is worth noting that only the matrix Λ needs to be determined k The value of the 10 elements of the upper triangle of (c) can determine the value of the entire matrix.
Further, based on the data index correlation judgment matrix Λ k Determining data index related construction vector beta k ，β k Is represented by the following formula:
further, a construction vector beta is related based on the data index k Determining a subjective data index weight vector through a data weight vector construction algorithm; a data weight vector construction algorithm, represented by the following formula:
likewise, the determining process of the subjective data index weight vector needs to be evaluated for rationality, which specifically includes: calculating a data consistency value of a data index correlation judgment matrix and a subjective data index weight vector through a preset data consistency index calculation formula; based on the data consistency value, determining the data consistency ratio of the data index correlation judgment matrix and the subjective data index weight vector, and determining that the subjective data index weight vector passes the rationality evaluation under the condition that the data consistency ratio is smaller than a preset threshold value, otherwise, correcting the data index correlation judgment matrix to redetermine the subjective data index weight vector.
In one implementation of the present application, the data consistency index calculation formula is represented by the following formula:
wherein ,CIk As an index of the consistency of the data,
for subjective data index weight vector, Λ k For the data index correlation judgment matrix,/a>
The i-th data index weight coefficient in the data index weight vector;
the data consistency ratio is calculated by the following formula:
wherein ,CRk Data consistency ratio, CI k Is a data consistency index.
In one embodiment of the present application, after determining the subjective data index weight vector, it is further required to determine an objective data index weight vector corresponding to the data index system to be tested based on measured data obtained by performing a performance test on the database to be tested.
Specifically, using a Jmeter to test the database object to be tested according to 4 performances in the performance index system I to be tested, and respectively collecting the data index system to be tested
And construct a measured data matrix:
wherein ,Ξk Representing a matrix of measured data,
representing database object D to be tested i In terms of performance index I k Data index corresponding to the execution time->
Is a function of the measured data of the test piece.
Further, the measured data matrix is subjected to standardized transformation to obtain a corresponding standardized matrix:
wherein ,Θk Is a standardized matrix.
Further, based on the normalized matrix, a corresponding index specific gravity matrix is determined:
wherein ,Ψk Is an index specific gravity matrix;
further, determining the information entropy of the data index through a preset data index information entropy calculation formula based on the index proportion matrix; wherein, the data index information entropy calculation formula is represented by the following formula:
wherein ,
entropy of data index, if there is +.>
Let ∈>
Further, based on the information entropy of the data index, determining an objective data index weight vector corresponding to the data index system to be tested through a preset objective data index weight vector calculation formula; wherein, the objective data index weight vector calculation formula is represented by the following formula:
it will be appreciated that the number of components,
is the performance index I k Corresponding data index system to be tested->
And a corresponding objective data index weight vector.
In one embodiment of the present application, after determining the subjective data index weight vector and the objective data index weight vector, the data index weight vector corresponding to the data index system to be measured is determined by the following formula:
In one embodiment of the application, after determining the performance index weight vector and the data index weight vector, determining a performance evaluation index corresponding to each performance index in the performance index system to be tested based on the data index weight vector, and determining a comprehensive evaluation index of the database to be tested based on the performance evaluation index and the performance index weight vector; wherein, the performance evaluation index corresponding to the performance index is determined by the following formula:
wherein ,p(Ik ,D i ) For the database D to be tested i Performance index I of (2) k A corresponding performance evaluation index;
is the performance index I k Corresponding data index weight vector W k The j-th data index weight coefficient of (a);
the comprehensive evaluation index of the database to be tested is determined by the following formula:
wherein ,p(Di ) Represents the comprehensive evaluation index, w, corresponding to the ith database to be tested j And the j-th performance index weight coefficient in the performance index weight vector is used as the j-th performance index weight coefficient.
The foregoing is a method embodiment presented herein. Based on the same inventive concept, the embodiment of the application also provides an evaluation device for database performance test, and the structure of the evaluation device is shown in fig. 2.
Fig. 2 is a schematic diagram of an internal structure of an evaluation device for database performance test according to an embodiment of the present application. As shown in fig. 2, the apparatus includes:
at least one processor 201;
and a memory 202 communicatively coupled to the at least one processor;
wherein the memory 202 stores instructions executable by the at least one processor, the instructions being executable by the at least one processor 201 to enable the at least one processor 201 to:
determining an evaluation object set corresponding to a database to be tested, and determining a performance index system to be tested and a data index system to be tested based on test requirements; the performance index system to be tested comprises a performance index for evaluating the database to be tested; the data index system to be tested comprises data indexes for evaluating single performance; the performance indexes comprise: query performance, delete performance, update performance, insert performance; the data index includes: average response time, average throughput, average CPU utilization, average memory occupancy, and anomaly rate;
based on a preset performance index correlation judgment matrix, determining a performance index weight vector corresponding to a performance index system to be tested through a preset performance weight vector construction algorithm;
Based on a preset data index correlation judgment matrix, determining a subjective data index weight vector corresponding to a data index system to be tested through a preset data weight vector construction algorithm, and determining an objective data index weight vector corresponding to the data index system to be tested based on measured data obtained by performing performance test on a database to be tested;
determining a data index weight vector corresponding to the data index system to be tested based on the subjective data index weight vector and the objective data index weight vector;
based on the data index weight vector, determining a performance evaluation index corresponding to each performance index in the performance index system to be tested, and based on the performance evaluation index and the performance index weight vector, determining a comprehensive evaluation index of the database to be tested.
Some embodiments of the present application provide a non-volatile computer storage medium corresponding to the evaluation of FIG. 1 for database performance testing, storing computer-executable instructions configured to:
determining an evaluation object set corresponding to a database to be tested, and determining a performance index system to be tested and a data index system to be tested based on test requirements; the performance index system to be tested comprises a performance index for evaluating the database to be tested; the data index system to be tested comprises data indexes for evaluating single performance; the performance indexes comprise: query performance, delete performance, update performance, insert performance; the data index includes: average response time, average throughput, average CPU utilization, average memory occupancy, and anomaly rate;
Based on a preset performance index correlation judgment matrix, determining a performance index weight vector corresponding to a performance index system to be tested through a preset performance weight vector construction algorithm;
based on a preset data index correlation judgment matrix, determining a subjective data index weight vector corresponding to a data index system to be tested through a preset data weight vector construction algorithm, and determining an objective data index weight vector corresponding to the data index system to be tested based on measured data obtained by performing performance test on a database to be tested;
determining a data index weight vector corresponding to the data index system to be tested based on the subjective data index weight vector and the objective data index weight vector;
based on the data index weight vector, determining a performance evaluation index corresponding to each performance index in the performance index system to be tested, and based on the performance evaluation index and the performance index weight vector, determining a comprehensive evaluation index of the database to be tested.
All embodiments in the application are described in a progressive manner, and identical and similar parts of all embodiments are mutually referred, so that each embodiment mainly describes differences from other embodiments. In particular, for the internet of things device and the medium embodiment, since they are substantially similar to the method embodiment, the description is relatively simple, and the relevant points are referred to in the description of the method embodiment.
The systems and media and the methods provided in the embodiments of the present application are in one-to-one correspondence, so that the systems and media also have similar beneficial technical effects to the corresponding methods, and since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the systems and media are not described here again.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and changes may be made to the present application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. which are within the spirit and principles of the present application are intended to be included within the scope of the claims of the present application.
Claims (10)
1. An evaluation method for database performance test, the method comprising:
determining an evaluation object set corresponding to a database to be tested, and determining a performance index system to be tested and a data index system to be tested based on test requirements; wherein the performance index system to be tested comprises performance indexes for evaluating a database to be tested; the data index system to be tested comprises data indexes for evaluating single performance; the performance index comprises: query performance, delete performance, update performance, insert performance; the data index comprises: average response time, average throughput, average CPU utilization, average memory occupancy, and anomaly rate;
based on a preset performance index correlation judgment matrix, determining a performance index weight vector corresponding to a performance index system to be tested through a preset performance weight vector construction algorithm;
Based on a preset data index correlation judgment matrix, determining a subjective data index weight vector corresponding to a data index system to be tested through a preset data weight vector construction algorithm, and determining an objective data index weight vector corresponding to the data index system to be tested based on measured data obtained by performing performance test on a database to be tested;
determining a data index weight vector corresponding to the data index system to be tested based on the subjective data index weight vector and the objective data index weight vector;
and determining a performance evaluation index corresponding to each performance index in the performance index system to be tested based on the data index weight vector, and determining a comprehensive evaluation index of the database to be tested based on the performance evaluation index and the performance index weight vector.
2. The method for evaluating a database performance test according to claim 1, wherein before determining a performance index weight vector corresponding to a performance index system to be tested by a preset performance weight vector construction algorithm based on a preset performance index correlation judgment matrix, the method further comprises:
determining the relative importance value of each performance index relative to other performance indexes based on a preset judgment matrix element value table;
Based on the relative importance value, constructing the performance index correlation judgment matrix, which is expressed by the following formula:
wherein lambda is a performance index correlation judgment matrix lambda i,j Representing performance index I i For performance index I j Is of relative importance, lambda i,j λ j,i ＝1(,j＝1,2,3,4)。
3. The method of claim 2, wherein the performance weight vector construction algorithm is represented by the following formula:
wherein W is a performance index weight vector, alpha is a construction vector, and alpha is expressed by the following formula:
4. a method of evaluating a database performance test according to claim 3, wherein after determining a performance index weight vector corresponding to a performance index system to be tested, the method further comprises:
performing rationality evaluation on the performance index weight vector, specifically including:
calculating the consistency value of the performance index correlation judgment matrix and the performance index weight vector through a preset consistency index calculation formula;
and determining the consistency ratio of the performance index correlation judgment matrix and the performance index weight vector based on the consistency value, and determining that the performance index weight vector passes rationality evaluation under the condition that the consistency ratio is smaller than a preset threshold value.
5. The method of evaluating a database performance test according to claim 4, wherein the consistency index calculation formula is expressed by the following formula:
wherein CI is a consistency index, W is a performance index weight vector, Λ is a performance index correlation judgment matrix, and W i The ith performance index weight coefficient in the performance index weight vector is used as the ith performance index weight coefficient;
the consistency ratio is calculated from the following formula:
wherein CR is a consistency ratio, and CI is a consistency index.
6. The method for evaluating a database performance test according to claim 1, wherein determining an objective data index weight vector corresponding to a data index system to be tested based on measured data obtained by performing the performance test on the database to be tested, specifically comprises:
based on measured data obtained by performance test of a database to be tested, constructing a measured data matrix;
performing standardized transformation on the measured data matrix to obtain a corresponding standardized matrix, and determining a corresponding index specific gravity matrix based on the standardized matrix;
determining the information entropy of the data index through a preset data index information entropy calculation formula based on the index proportion matrix;
And determining an objective data index weight vector corresponding to the data index system to be tested through a preset objective data index weight vector calculation formula based on the information entropy of the data index.
7. The method of evaluating a database performance test according to claim 6, wherein the standardized transformation of the measured data matrix is implemented by the following formula:
wherein ,Θk In order to normalize the matrix,
representing database object D to be tested i In terms of performance index I k Data index corresponding to the execution time->
Is a measured data of (1); />
Performance index I k The corresponding measured data matrix is represented by the following formula:
wherein ,Ξk Representing the measured data matrix;
the index specific gravity matrix is determined by the following formula:
wherein ,Ψk Is an index specific gravity matrix;
the data index information entropy calculation formula is represented by the following formula:
wherein ,
entropy of data index, if there is +.>
Let ∈>
8. The method of claim 7, wherein the performance evaluation index corresponding to the performance index is determined by the following formula:
wherein ,p(Ik I) is the database D to be tested i Performance index I of (2) k A corresponding performance evaluation index;
is the performance index I k Corresponding data index weight vector W k The j-th data index weight coefficient of (a);
the comprehensive evaluation index of the database to be tested is determined by the following formula:
wherein ,p(Di ) Represents the comprehensive evaluation index, w, corresponding to the ith database to be tested j And the j-th performance index weight coefficient in the performance index weight vector is used as the j-th performance index weight coefficient.
9. An evaluation device for database performance testing, the device comprising:
at least one processor;
and a memory communicatively coupled to the at least one processor;
wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to:
determining an evaluation object set corresponding to a database to be tested, and determining a performance index system to be tested and a data index system to be tested based on test requirements; wherein the performance index system to be tested comprises performance indexes for evaluating a database to be tested; the data index system to be tested comprises data indexes for evaluating single performance; the performance index comprises: query performance, delete performance, update performance, insert performance; the data index comprises: average response time, average throughput, average CPU utilization, average memory occupancy, and anomaly rate;
Based on a preset performance index correlation judgment matrix, determining a performance index weight vector corresponding to a performance index system to be tested through a preset performance weight vector construction algorithm;
based on a preset data index correlation judgment matrix, determining a subjective data index weight vector corresponding to a data index system to be tested through a preset data weight vector construction algorithm, and determining an objective data index weight vector corresponding to the data index system to be tested based on measured data obtained by performing performance test on a database to be tested;
determining a data index weight vector corresponding to the data index system to be tested based on the subjective data index weight vector and the objective data index weight vector;
and determining a performance evaluation index corresponding to each performance index in the performance index system to be tested based on the data index weight vector, and determining a comprehensive evaluation index of the database to be tested based on the performance evaluation index and the performance index weight vector.
10. A non-volatile computer storage medium storing computer executable instructions for evaluation of database performance testing, the computer executable instructions configured to:
Determining an evaluation object set corresponding to a database to be tested, and determining a performance index system to be tested and a data index system to be tested based on test requirements; wherein the performance index system to be tested comprises performance indexes for evaluating a database to be tested; the data index system to be tested comprises data indexes for evaluating single performance; the performance index comprises: query performance, delete performance, update performance, insert performance; the data index comprises: average response time, average throughput, average CPU utilization, average memory occupancy, and anomaly rate;
based on a preset performance index correlation judgment matrix, determining a performance index weight vector corresponding to a performance index system to be tested through a preset performance weight vector construction algorithm;
based on a preset data index correlation judgment matrix, determining a subjective data index weight vector corresponding to a data index system to be tested through a preset data weight vector construction algorithm, and determining an objective data index weight vector corresponding to the data index system to be tested based on measured data obtained by performing performance test on a database to be tested;
determining a data index weight vector corresponding to the data index system to be tested based on the subjective data index weight vector and the objective data index weight vector;
And determining a performance evaluation index corresponding to each performance index in the performance index system to be tested based on the data index weight vector, and determining a comprehensive evaluation index of the database to be tested based on the performance evaluation index and the performance index weight vector.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310078383.3A CN116166513A (en) | 2023-01-30 | 2023-01-30 | Evaluation method, device and storage medium for database performance test |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310078383.3A CN116166513A (en) | 2023-01-30 | 2023-01-30 | Evaluation method, device and storage medium for database performance test |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116166513A true CN116166513A (en) | 2023-05-26 |
Family
ID=86415950
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310078383.3A Pending CN116166513A (en) | 2023-01-30 | 2023-01-30 | Evaluation method, device and storage medium for database performance test |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116166513A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116922397A (en) * | 2023-09-13 | 2023-10-24 | 成都明途科技有限公司 | Robot intelligent level measuring method and device, robot and storage medium |
| CN117056182A (en) * | 2023-07-13 | 2023-11-14 | 北京新数科技有限公司 | SQL Server database performance evaluation method |
-
2023
- 2023-01-30 CN CN202310078383.3A patent/CN116166513A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117056182A (en) * | 2023-07-13 | 2023-11-14 | 北京新数科技有限公司 | SQL Server database performance evaluation method |
| CN117056182B (en) * | 2023-07-13 | 2024-05-03 | 北京新数科技有限公司 | SQL SERVER database performance evaluation method |
| CN116922397A (en) * | 2023-09-13 | 2023-10-24 | 成都明途科技有限公司 | Robot intelligent level measuring method and device, robot and storage medium |
| CN116922397B (en) * | 2023-09-13 | 2023-11-28 | 成都明途科技有限公司 | Robot intelligent level measuring method and device, robot and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116166513A (en) | Evaluation method, device and storage medium for database performance test | |
| US7747987B1 (en) | System and method of analyzing risk in risk-based software testing | |
| CN107168995B (en) | Data processing method and server | |
| CN108241901B (en) | Transformer early warning evaluation method and device based on prediction data | |
| Yu et al. | Experience in predicting fault-prone software modules using complexity metrics | |
| CN105468510A (en) | Method and system for evaluating and tracking software quality | |
| EP4075281A1 (en) | Ann-based program test method and test system, and application | |
| CN111881058B (en) | Software engineering quality prediction method | |
| CN110377519B (en) | Performance capacity test method, device and equipment of big data system and storage medium | |
| JP2023511464A (en) | SYSTEM, METHOD, AND COMPUTER-READABLE MEDIUM FOR ANALYSIS OF PRODUCT FAILURE FACTORS | |
| CN114971891A (en) | Risk prediction method and device, processor and electronic equipment | |
| Saha et al. | Acceptance sampling inspection plan for the Lindley and power Lindley distributed quality characteristics | |
| CN113408963A (en) | Textile yarn quality rating method and device, storage medium and processor | |
| JP7231829B2 (en) | Machine learning program, machine learning method and machine learning apparatus | |
| CN116401232A (en) | Database parameter configuration optimization method and device, electronic equipment and storage medium | |
| Elsayed et al. | Optimal design of proportional hazards based accelerated life testing plans | |
| CN115878986A (en) | Degradation trend evaluation method for variable working condition bearing | |
| Yan et al. | Revisiting the correlation between alerts and software defects: A case study on myfaces, camel, and cxf | |
| CN114139625A (en) | Part quality classification method and system based on degradation trajectory curve analysis | |
| Aslam et al. | Time truncated life tests using the generalized multiple dependent state sampling plans for various life distributions | |
| CN114330818A (en) | Dynamic water demand prediction method based on main driving factor screening and deep learning | |
| CN112116139A (en) | Power demand prediction method and system | |
| CN111523685A (en) | Method for reducing performance modeling overhead based on active learning | |
| WO2018177293A1 (en) | Sample-based multidimensional data cloning | |
| CN110647466B (en) | Program quality supervision method and device based on DevOps |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |