CN114996112A - System performance evaluation method and device, storage medium and electronic equipment - Google Patents
System performance evaluation method and device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN114996112A CN114996112A CN202210739958.7A CN202210739958A CN114996112A CN 114996112 A CN114996112 A CN 114996112A CN 202210739958 A CN202210739958 A CN 202210739958A CN 114996112 A CN114996112 A CN 114996112A
- Authority
- CN
- China
- Prior art keywords
- transaction type
- online processing
- performance
- evaluation
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
- G06F11/3495—Performance evaluation by tracing or monitoring for systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3409—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
- G06F11/3419—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment by assessing time
Abstract
The invention provides a method and a device for evaluating system performance, a storage medium and electronic equipment, wherein the method comprises the following steps: determining data sampling time, and acquiring each data request of the system in the data sampling time; determining the transaction type of each data request; determining the request proportion of each transaction type based on the number of each data request of each transaction type, and determining the response time coefficient of the system by applying each request proportion; determining the online processing evaluation value of the system for each transaction type by applying the response time coefficient; and calculating each online processing evaluation value to obtain an online processing fluctuation range, and evaluating the performance of the system by using the online processing fluctuation range. The invention introduces the concept of response time coefficient, can accurately calculate the online processing fluctuation range of the system, reduces the manual participation in the whole process, reduces the workload of workers, reduces the time cost spent on evaluation, and effectively improves the efficiency of evaluating the performance of the system.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a method and an apparatus for evaluating system performance, a storage medium, and an electronic device.
Background
The big data system is applied to various aspects in life, and enterprises in various fields apply the big data system so as to facilitate the realization of business and the expansion of work. The big data system processes data which is the core work of the system, when the system processes data at present, the data is processed on line, the processing process needs to be processed on line with other systems, and the whole process involves large data volume and high complexity.
The performance of the system is very tested by processing data with large data volume and high complexity, and in order to ensure the stable operation of the system, the performance of the system needs to be evaluated so that operation and maintenance personnel can maintain the system in time. At present, the performance of the system is usually evaluated by an operation and maintenance person by processing various performance data of the system, and then the performance of the system is evaluated.
Disclosure of Invention
In view of this, the present invention provides a method and an apparatus for evaluating system performance, a storage medium, and an electronic device, which reduce the participation of workers in the process of evaluating system performance, shorten the time required for evaluating system performance, and improve evaluation efficiency.
In order to achieve the above purpose, the embodiments of the present invention provide the following technical solutions:
the invention discloses a method for evaluating system performance in a first aspect, which comprises the following steps:
determining data sampling time, and acquiring each data request of a system in the data sampling time;
determining a transaction type to which each data request belongs;
determining a request proportion of each transaction type based on the number of each data request of each transaction type, and determining a response time coefficient of the system by applying each request proportion;
applying the response time coefficient to determine an online processing evaluation value of the system for each transaction type;
and calculating each online processing evaluation value to obtain an online processing fluctuation range, and evaluating the performance of the system by using the online processing fluctuation range.
Optionally, the method for determining the response time coefficient of the system by applying each request duty includes:
acquiring the complexity of each transaction type;
calculating the complexity and the request proportion of each transaction type to obtain a response coefficient of each transaction type;
and summing the response coefficients to obtain the response time coefficient of the system.
The method described above, optionally, the determining an online processing evaluation value of the system for each transaction type by applying the response time coefficient, includes:
determining a response time for each data request;
for each transaction type, determining an arithmetic average of the query rate per second for that transaction type based on the response time of each data request belonging to that transaction type;
and processing the response time coefficient, the arithmetic mean value of each transaction type and a preset response time index value to obtain an online processing evaluation value of each transaction type.
Optionally, the above method, where the operation is performed on each online processing evaluation value to obtain an online processing fluctuation range, includes:
carrying out averaging processing on each online processing evaluation value to obtain an online processing evaluation average value;
and processing each online processing evaluation value and the online processing evaluation average value based on a preset operation mode to obtain an online processing fluctuation range.
The method described above, optionally, the evaluating the performance of the system using the online processing fluctuation range, includes:
in the preset performance dimensions, marking the performance dimension to which the online processing fluctuation range belongs as a first target performance dimension, and marking the remaining performance indexes as second target performance dimensions;
marking the evaluation indexes corresponding to the online processing fluctuation range in the first target performance dimension, and acquiring index values of all unmarked evaluation indexes in the first target performance dimension;
and generating an evaluation result of the system in the first target performance dimension based on the online processing fluctuation range and each index value.
The above method, optionally, further includes:
determining each evaluation index of each second target performance dimension;
acquiring an index value of each evaluation index of each second target performance dimension;
and for each second target performance dimension, generating an evaluation result of the system in the second target performance dimension based on the index values of the evaluation indexes of the second target performance dimension.
The second aspect of the present invention discloses an apparatus for evaluating system performance, comprising:
the system comprises an acquisition unit, a data processing unit and a data processing unit, wherein the acquisition unit is used for determining data sampling time and acquiring each data request of the system in the data sampling time;
a first determining unit, configured to determine a transaction type to which each of the data requests belongs;
a second determining unit, configured to determine a request proportion for each transaction type based on the number of data requests for each transaction type, and determine a response time coefficient of the system by applying each request proportion;
a third determining unit, configured to apply the response time coefficient to determine an online processing evaluation value of the system for each transaction type;
and the evaluation unit is used for calculating each online processing evaluation value to obtain an online processing fluctuation range, and evaluating the performance of the system by using the online processing fluctuation range.
The above apparatus, optionally, the second determining unit includes:
an obtaining subunit, configured to obtain complexity of each transaction type;
the operation subunit is used for operating the complexity and the request duty ratio of each transaction type to obtain a response coefficient of each transaction type;
and the summation processing subunit is used for carrying out summation processing on each response coefficient to obtain a response time coefficient of the system.
The above apparatus, optionally, the third determining unit includes:
a first determining subunit, configured to determine a response time of each data request;
a second determining subunit, configured to determine, for each transaction type, an arithmetic average of the query rate per second for the transaction type based on a response time of each data request belonging to the transaction type;
and the first obtaining subunit is used for processing the response time coefficient, the arithmetic mean value of each transaction type and a preset response time index value to obtain an online processing evaluation value of each transaction type.
The above apparatus, optionally, the evaluation unit, includes:
the averaging processing subunit is used for carrying out averaging processing on each online processing evaluation value to obtain an online processing evaluation average value;
and the second obtaining subunit is used for processing each online processing evaluation value and the online processing evaluation average value based on a preset operation mode to obtain an online processing fluctuation range.
The above apparatus, optionally, the evaluation unit includes:
the first marking subunit is used for marking the performance dimension to which the online processing fluctuation range belongs as a first target performance dimension in all preset performance dimensions, and marking all the remaining performance indexes as second target performance dimensions;
a second marking subunit, configured to mark an evaluation index corresponding to the online processing fluctuation range in the first target performance dimension, and obtain an index value of each unmarked evaluation index in the first target performance dimension;
and the generation subunit is used for generating an evaluation result of the system in the first target performance dimension based on the online processing fluctuation range and each index value.
The above apparatus, optionally, further comprises:
a fourth determining unit, configured to determine evaluation indexes of each of the second target performance dimensions;
an acquisition unit configured to acquire an index value of each evaluation index of each of the second target performance dimensions;
and the generating unit is used for generating an evaluation result of the system in the second target performance dimension based on the index values of the evaluation indexes of the second target performance dimension for each second target performance dimension.
In a third aspect, the present invention discloses a storage medium, which includes stored instructions, wherein when the instructions are executed, a device on which the storage medium is located is controlled to execute the above-mentioned method for evaluating system performance.
In a fourth aspect, the present invention discloses an electronic device comprising a memory, and one or more instructions, wherein the one or more instructions are stored in the memory and configured to be executed by the one or more processors to perform the method for evaluating system performance as described above.
Compared with the prior art, the invention has the following advantages:
the invention provides a method and a device for evaluating system performance, a storage medium and electronic equipment, wherein the method comprises the following steps: determining data sampling time, and acquiring each data request of the system in the data sampling time; determining the transaction type of each data request; determining the request proportion of each transaction type based on the number of each data request of each transaction type, and determining the response time coefficient of the system by applying each request proportion; determining the online processing evaluation value of the system for each transaction type by applying the response time coefficient; and calculating each online processing evaluation value to obtain an online processing fluctuation range, and evaluating the performance of the system by using the online processing fluctuation range. The method introduces the concept of response time coefficient, can accurately calculate the online processing fluctuation range of the system, reduces the manual participation in the whole process, reduces the workload of workers, reduces the time cost spent on evaluation, and effectively improves the efficiency of evaluating the performance of the system; and, the performance of the system is evaluated by using the online processing fluctuation range, so that the accuracy of the evaluation of the system performance can be improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a flowchart of a method for evaluating system performance according to an embodiment of the present invention;
fig. 2 is a flowchart of a method for determining a response time coefficient of a system by applying each request duty according to an embodiment of the present invention;
FIG. 3 is a flowchart of a method for determining an online processing evaluation value for each transaction type by using a response time coefficient according to an embodiment of the present invention;
FIG. 4 is a flowchart of a method for evaluating system performance using online process fluctuation range, according to an embodiment of the present invention;
FIG. 5 is a schematic structural diagram of an apparatus for evaluating system performance according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In this application, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Interpretation of terms:
online transaction processing (OLTP): the On-Line Transaction Processing, also called Real Time System (Real Time System), supports quick response and large concurrency of transactions, and the current running state of an enterprise, completes database application of daily tasks contained in enterprise management, and generally has no complex query and analysis Processing.
Online analytical processing (OLAP): an On-Line Analytical Processing (also called a Decision Support System, DSS) is a main application form of a data warehouse System, and is specially designed to Support complex analysis operations and to emphasize Decision Support for Decision-making personnel and high-level management personnel.
Big data: big data set containing big data set with big volume, multiple sources, fast generation, diversity and other characteristics and difficult to effectively process by traditional data architecture.
Performance indexes are as follows: a set of parameters that can be used to evaluate the performance of an application system.
Processing capacity: and starting timing from the time when the client sends a request to the server, and ending timing after receiving a response of the server, so as to calculate the used time and the number of completed transactions.
Response time coefficient: and defining an index item representing the characteristics of the big data according to the complexity of the online analysis processing.
Fluctuation range: the distance between the peak and the trough formed by repeatedly fluctuating processing capacity within a certain period is calculated as the ratio of standard deviation to mean value.
The traditional relational database application belongs to on-line transaction processing, is basic and daily transaction processing, records instant addition, deletion, modification and check, and consists of short atomic transactions, for example, a transaction is a money access in a bank. An important performance requirement for an online transaction processing system is a performance index, which is specifically embodied as a real-Time Response Time (Response Time), i.e., the Time required from a user to a server to respond to a request after the user sends data to the terminal. The other is processing capacity, i.e., the number of transactions processed per unit time, tps (transaction per second).
The big data processing is the core application of a data warehouse, is an online analysis processing transaction, supports complex analysis operation, and is characterized in that the data volume is large, and a user can obtain the desired information only by counting mass data; the requirement on real-time performance is not high, the decision support is emphasized, dynamic query is generally realized, and intuitive and understandable query results can be provided. A typical application is a complex dynamic reporting system. The online transaction processing time of the system is long and unstable, which causes the unstable performance of the system and affects the capability of the system for processing data, so that the performance of the system needs to be evaluated and the system needs to be maintained in time.
In addition to the fact that the conventional method for evaluating the performance of the system requires a large investment and is not efficient, it is difficult to determine the on-line analysis processing capability and fluctuation range of the system by using the conventional method for evaluating the performance of the system, which is a problem to be solved urgently.
The invention is operational with numerous general purpose or special purpose computing device environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet-type devices, multi-processor apparatus, distributed computing environments that include any of the above devices or equipment, and the like. The invention can be applied to an online system or a data system which can process various data, and the online system and the data system are both formed by using a computer terminal or data processing equipment.
Referring to fig. 1, a flowchart of a method for evaluating system performance according to an embodiment of the present invention is specifically described as follows:
s101, determining data sampling time and acquiring each data request of the system in the data sampling time.
And receiving a system evaluation instruction, analyzing the system evaluation instruction, and acquiring data sampling time in the system evaluation instruction.
The system evaluation instruction is an instruction generated when a worker needs to evaluate the performance of the system, and the system evaluation instruction includes, but is not limited to, information such as data sampling time, system identity identification, instruction generation time and the like.
The data sampling time staff can set according to actual requirements, for example, the data sampling time can be the previous 3 days, the previous 5 days, and can also be a specific date.
The data requests of the system within the data sampling time are collected in the historical database of the system, and it should be noted that the data requests are requests processed by the system within the data sampling time.
S102, determining the transaction type of each data request.
It should be noted that the system can process requests of various transaction types, for example, the transaction types include but are not limited to online processing type, business processing type, and the like.
There is a corresponding transaction type for each data request.
S103, determining the request ratio of each transaction type based on the number of each data request of each transaction type, and determining the response time coefficient of the system by applying each request ratio.
Furthermore, the request proportion of the transaction type is the proportion of the number of the data requests of the transaction type in the total data requests; illustratively, counting the total number of each data request; determining the number of data requests of each transaction type; for each transaction type, dividing the number of the data requests of the transaction type by the total number of the data requests to obtain the request proportion of the transaction type.
It should be noted that the request fraction of the transaction type may also be referred to as an online analysis transaction fraction, specifically, the fraction of the total number of requests of the transaction in the analysis range to be examined is within the whole examination range.
Using the respective request ratios, the response time factor of the system during the data sampling time is determined, it being noted that the response time factor is related to the number of transaction types provided during the data sampling time.
And S104, applying the response time coefficient to determine the online processing evaluation value of the system for each transaction type.
The online processing evaluation value is an evaluation value of the online processing process of the system when the system processes the request of the transaction type, and the online processing fluctuation range of the system can be calculated by using the evaluation value.
And S105, calculating each online processing evaluation value to obtain an online processing fluctuation range, and evaluating the performance of the system by using the online processing fluctuation range.
In the method provided by the embodiment of the invention, data sampling time is determined, and each data request of a system in the data sampling time is acquired; determining the transaction type of each data request; determining the request proportion of each transaction type based on the number of each data request of each transaction type, and determining the response time coefficient of the system by applying each request proportion; determining the online processing evaluation value of the system for each transaction type by applying the response time coefficient; and calculating each online processing evaluation value to obtain an online processing fluctuation range, and evaluating the performance of the system by using the online processing fluctuation range. The method introduces the concept of response time coefficient, can accurately calculate the online processing fluctuation range of the system, reduces the manual participation in the whole process, reduces the workload of workers, reduces the time cost spent on evaluation, and effectively improves the efficiency of evaluating the performance of the system; and, the performance of the system is evaluated by using the online processing fluctuation range, so that the accuracy of the evaluation of the system performance can be improved.
Referring to fig. 2, a flowchart of a method for determining a response time coefficient of a system by applying each request duty according to an embodiment of the present invention is specifically described as follows:
s201, acquiring the complexity of each transaction type.
And determining the complexity of each transaction type based on a preset transaction complexity data table.
The transaction complexity data table contains the complexity of each transaction type that can be processed by the system, and it should be noted that the complexity of a transaction type can be understood as the complexity of processing logic of a transaction type, and the complexity can be set according to various complexity indexes such as data volume, the number of database tables, data dimensions, data summarization times, complex derivative data and the like involved in the processing logic of the transaction type.
The data volume can be in the order of one hundred thousand, millions and more; the number of the database tables can be divided into 1, 3, 5 and more than 5; the data dimension can be divided into two dimensions, three dimensions, four dimensions, five dimensions and more; the data summarization times can be divided into 1 time, 2 times, 3 times, 4 times and more; complex derivative data can be classified as yes or no (e.g., periodic comparison, share calculation, equal mean square error, etc.); further, different values of the complexity index correspond to different complexities, and referring to table 1, the example in the table shows the complexity of various complexity indexes under different values.
It should be noted that the data size, the number of the base tables, the data latitude, the summary number and the derivative data in table 1 are all complexity indexes, wherein the level definition represents a specific numerical value of the complexity index.
Further, the complexity of the transaction type is the sum of the complexity corresponding to the data volume, the complexity corresponding to the number of the library tables, the complexity corresponding to the data latitude, the complexity corresponding to the summary number, and the complexity corresponding to the derivative data, for example, if the data volume of the transaction type 1 is hundred thousand, the number of the library tables is 1, the data dimension is two-dimensional, the summary number is 1, and the derivative data is no, the complexity of the transaction type 1 is 5.
TABLE 1
S202, calculating the complexity and the request proportion of each transaction type to obtain the response coefficient of each transaction type.
It should be noted that, for each transaction type, the response coefficient of the transaction type can be obtained by multiplying the complexity of the transaction type by the request duty ratio; illustratively, the request duty ratio of 1 for transaction type 1 is multiplied by the complexity 1 to obtain a response coefficient of 1.
And S203, summing the response coefficients to obtain the response time coefficient of the system.
For example, the response time coefficient 1+ response coefficient 2+ · · · response coefficient n + request complexity 1+ complexity 2 · · complexity · · ± request complexity n; thereby, the response time coefficient of the system can be obtained.
In the method provided by the embodiment of the invention, the response time coefficient of the system can be calculated by applying the complexity and the request duty ratio of each transaction type, the response time coefficient is determined based on each transaction type related to the system in the data sampling time, and the response time coefficient is dynamic, so that the performance of the system can be dynamically evaluated.
Referring to fig. 3, a flowchart of a method for determining an online processing evaluation value of each transaction type by a system using a response time coefficient according to an embodiment of the present invention is specifically described as follows:
s301, determining the response time of each data request.
The response time is the total time taken to perform the data request from the beginning until the response data is received.
S302, for each transaction type, determining an arithmetic mean of the query rate per second for the transaction type based on the response time of each data request belonging to the transaction type.
It should be noted that the query rate per second can be expressed using QPS, which is the arithmetic average of QPS in computing transaction typesWhen averaging, the QPS for each data request of the transaction type needs to be calculated first, and further, the QPS for each data request of the transaction type may form a set q (q) 1 ,q 2 ,...,q n ) Wherein q is 1 QPS, q for transaction type data request 1 2 The QPS for data request 2 is of transaction type, and so on, and will not be described here.
Further, grouping QPSs of data requests of each transaction type according to a response time coefficient and a response time index value, and calculating the arithmetic mean of the query rate per second of the transaction type; it should be noted that, the response time index value is a value set in advance for the transaction type, and the response time index values for different transaction types are different.
Further, QPS for various transaction types may constitute a set Q (Q) 1 ,Q 2 ,...,Q n ) Wherein Q is 1 Arithmetic mean, Q, representing transaction type 1 2 Represents the arithmetic mean of transaction type 2, and so on, and will not be described in detail here.
S303, processing the response time coefficient, the arithmetic mean value of each transaction type and a preset response time index value to obtain an online processing evaluation value of each transaction type.
For each transaction type, the response time coefficient, the arithmetic mean of the transaction type and the response time index value are processed to obtain the online processing evaluation value of the transaction type, which is, for example, the arithmetic mean/(response time coefficient), further, the online processing evaluation value may be a value characterizing the online analysis processing capability (APT) of the system, the APT is the capability of the online analysis processing number completed in a certain time, the APT is related to the QPS, the response time system and the response time index value, preferably, the online processing evaluation values of the transaction types may constitute an array a, a is the number of concurrent requests/(response time system response time index value) ═ the arithmetic mean value of each set QPS/(response time coefficient response time index value) ═ Q/(response time coefficient response time).Inter-index value), where the number of concurrent requests may be the total number of data requests. Exemplary, A (A) 1 ,A 1 ,...,A n ) (ii) a Wherein, A 1 Evaluation value of online processing, A, representing transaction type 1 2 The online processing evaluation value of transaction type 2 is shown, and so on, and will not be described herein.
The response time index value is an expected value set according to the complexity of the transaction type, so that a request belonging to the transaction type can receive a response within a given value range.
Referring to table 2, a summary table of complexity, request duty and preset response time index values for each transaction type is provided.
| Type of transaction | Complexity of | Request to duty ratio | Response time indicator |
| 1 | Complexity 1 | Ratio of 1 | Index value 1 |
| 2 | Complexity 2 | Ratio of 2 | Index value 2 |
| ... | … | … | … |
| n | Complexity n | Ratio n | Index value n |
TABLE 2
Further, after the online processing evaluation value of each transaction type is determined, the online processing fluctuation range can be calculated by using the online processing evaluation value, and it should be noted that the online processing fluctuation range is a distance from the mean value, in which the online processing analysis capability of the system repeatedly fluctuates within a certain time. Further, the online processing fluctuation range is related to the APT, and the calculation process may be the APT standard deviation/the APT mean.
In the process of determining the online processing fluctuation range, firstly, carrying out averaging processing on each online processing evaluation value to obtain an online processing evaluation average value; processing each online processing evaluation value and each online processing evaluation average value based on a preset operation mode to obtain an online processing fluctuation range; further, the online processing fluctuation range may be the discrete coefficient of the array a, and further, the formula applied by the preset operation mode is:
wherein c is the online processing fluctuation range,
evaluation of the mean value for on-line processing, A 1 On-line processing fluctuation evaluation value for transaction type 1, A 2 The online processing fluctuation evaluation value for transaction type 2 is analogized, and will not be described herein.
In the process of evaluating the performance of the system, the concept of the response time coefficient is introduced, on the basis, the online analysis capability and the fluctuation range of the system are calculated, and the performance of the system can be evaluated quickly and accurately by using the calculation result.
Referring to fig. 4, a flowchart of a method for evaluating system performance using online processing fluctuation range according to an embodiment of the present invention is specifically described as follows:
s401, in the preset performance dimensions, marking the performance dimension to which the online processing fluctuation range belongs as a first target performance dimension, and marking the remaining performance indexes as second target performance dimensions.
The preset performance dimensions include, but are not limited to, a single transaction dimension, a mixed transaction processing capability dimension, and a stability dimension, and optionally, the performance dimension may also be referred to as a performance index.
Furthermore, the performance dimension to which the online processing fluctuation range belongs is a stability dimension, the stability dimension is further determined as a first target performance dimension, and both the single transaction dimension and the mixed transaction processing capacity dimension are determined as a second target performance dimension.
S402, marking the evaluation indexes corresponding to the online processing fluctuation range in the first target performance dimension, and acquiring index values of all unmarked evaluation indexes in the first target performance dimension.
Each performance dimension has a plurality of evaluation indexes, and for example, each evaluation index corresponding to the stability dimension includes, but is not limited to, a total transaction amount within a system stable operation duration, a transaction response time within the system stable operation duration, a transaction response time standard rate within the system stable operation duration, a stable operation transaction processing capacity fluctuation range, and a resource utilization rate — memory usage, and further, the stable operation transaction processing capacity fluctuation range is an online processing fluctuation range.
It should be noted that, when obtaining the index values of the unidentified and evaluated indexes in the first target performance dimension, the index values may be obtained from data corresponding to the evaluated indexes, for example, the resource utilization rate-memory usage may be obtained from memory data of the system, and the total transaction amount in the system stable operation duration may be obtained by statistics from data of the system stable operation.
And S403, generating an evaluation result of the system in the first target performance dimension based on the online processing fluctuation range and each index value.
It should be noted that each evaluation index has a corresponding judgment standard, and for each evaluation index, whether the evaluation index meets the judgment standard can be judged, so as to obtain a judgment result of the evaluation index; further, the evaluation result of the system in the first target performance dimension may be generated according to each judgment result.
S404, determining each evaluation index of each second target performance dimension.
S405, obtaining the index value of each evaluation index of each second target performance dimension.
In the process of obtaining the index value of each evaluation index, the index value can be extracted from data corresponding to the evaluation index in the system, and for example, the evaluation index of resource utilization rate-CPU utilization rate in the hybrid processing capacity can be extracted from CPU usage data of the system.
S406, for each second target performance dimension, generating an evaluation result of the system in the second target performance dimension based on the index values of the evaluation indexes of the second target performance dimension.
Further, each evaluation index of each second target performance dimension has a judgment standard, and for each evaluation index of each second target performance dimension, whether the index value of the evaluation index meets the judgment standard of the evaluation index can be judged, and a judgment result is generated; and for each second target performance dimension, generating an evaluation result according to each judgment result of the performance dimension.
Referring to table 3, the evaluation indexes of the performance dimensions, the definition of the index value of each evaluation index, and the judgment criteria of each evaluation index value provided by the embodiment of the present invention are summarized.
TABLE 3
Preferably, the requirement in the table may be a requirement for evaluating the performance of the system, and the determination criterion in the table is that it can be determined that the evaluation index satisfies the determination requirement after the index value satisfies the criterion, and further, when the number proportion of the evaluation indexes satisfying the requirement is smaller than the preset proportion, an evaluation result of unstable performance of the system may be generated, and when the number proportion of the evaluation indexes satisfying the requirement is greater than or equal to the preset proportion, an evaluation result of stable performance of the system may be generated.
Preferably, the stability of the system may be evaluated by using the fluctuation range alone, for example, when the fluctuation range is not within a preset range, an evaluation result of the system instability is generated, and when the fluctuation range is within a preset range, an evaluation result of the system stability is generated.
Further, in the process of practical application, the invention can use 4 modules to implement the scheme of the invention, specifically as follows:
the module 1 defines a performance requirement item, the performance requirement of the big data system is measured through a performance index (performance dimension), the performance index of the big data system is determined through defining the performance requirement item, a response time system is defined, and the content of the module is preset.
And the module 2 establishes the relation between the requirement items, further, the relation between the requirement items can refer to the content of the evaluation index in the table 3, and the content of the module is preset.
The module 3 is used for acquiring data and acquiring basic data such as the quantity of concurrent requests, response time and the like; further, the required data can be collected according to the requirement items established in the module 2.
And a module 4 for calculating QPS, APT and fluctuation range according to the data collected in the module 3.
According to the invention, the processing capacity and the fluctuation range are calculated according to the relation among the response time, the response time coefficient and the processing capacity, and the evaluation result of the online analysis processing capacity and the stability of the system is obtained.
The online transaction processing in the current system relies on a traditional relational database, aiming to allow the application program to immediately transmit the original data to the computing center for processing at any time, write or update only the required data, and respond in a very short time to give a result so as to process a single transaction as soon as possible. The performance requirement aspect is mature, and the measurement method has definite performance indexes including response time, processing capacity and definite calculation rules.
For the online analysis processing of big data, the application program processing depends on a data warehouse, the processing data reaches giga (P) bytes, the query requirement of a user is complex, and the query requirement involves not only querying or manipulating one or more records in one table, but also carrying out data analysis and information synthesis on the data of tens of millions of records in multiple tables. The performance requirements of the query are similar and different from those of the traditional online transaction processing.
With the rapid development of database technology, the performance requirements of online analysis processing are increasingly prominent in typical applications of large data systems, and from being ignored or weakened to the attention of practitioners and management layers, the performance requirements are urgent, but there is no clear statistical method or rule for judging the stability of online analysis.
Based on the above characteristics of the online transaction processing and the online analysis processing, the performance requirement of the online analysis processing is derived from the performance requirement of the online transaction processing, the index item is further expanded on the basis of the response time and the processing capacity of the performance index item of the online transaction processing, and the response time coefficient concept is provided for calculating the processing capacity and the fluctuation range of the online analysis transaction to judge the operation stability performance of the big data system.
In accordance with the method shown in FIG. 1, the present invention provides a system performance evaluation device, which is used to support the implementation of the method shown in FIG. 1, and is disposed in a big data system or an online processing system.
Referring to fig. 5, a schematic structural diagram of an apparatus for evaluating system performance according to an embodiment of the present invention is specifically described as follows:
the acquisition unit 501 is configured to determine data sampling time and acquire each data request of the system within the data sampling time;
a first determining unit 502, configured to determine a transaction type to which each of the data requests belongs;
a second determining unit 503, configured to determine a request proportion for each transaction type based on the number of data requests for each transaction type, and determine a response time coefficient of the system by applying each request proportion;
a third determining unit 504, configured to apply the response time coefficient to determine an online processing evaluation value of the system for each transaction type;
and the evaluation unit 505 is configured to calculate each online processing evaluation value to obtain an online processing fluctuation range, and evaluate the performance of the system using the online processing fluctuation range.
In the device provided by the embodiment of the invention, the data sampling time is determined, and each data request of the system in the data sampling time is acquired; determining the transaction type of each data request; determining the request proportion of each transaction type based on the number of each data request of each transaction type, and determining the response time coefficient of the system by applying each request proportion; determining the online processing evaluation value of the system for each transaction type by applying the response time coefficient; and calculating each online processing evaluation value to obtain an online processing fluctuation range, and evaluating the performance of the system by using the online processing fluctuation range. The method introduces the concept of response time coefficient, can accurately calculate the online processing fluctuation range of the system, reduces the manual participation in the whole process, reduces the workload of workers, reduces the time cost spent on evaluation, and effectively improves the efficiency of evaluating the performance of the system; and, the performance of the system is evaluated by using the online processing fluctuation range, so that the accuracy of the evaluation of the system performance can be improved.
In another embodiment provided by the present invention, the second determining unit 503 of the apparatus may be configured to:
an obtaining subunit, configured to obtain complexity of each transaction type;
the operation subunit is used for operating the complexity and the request duty ratio of each transaction type to obtain a response coefficient of each transaction type;
and the summation processing subunit is used for carrying out summation processing on each response coefficient to obtain a response time coefficient of the system.
In another embodiment provided by the present invention, the third determining unit 504 of the apparatus may be configured to:
a first determining subunit, configured to determine a response time of each data request;
a second determining subunit operable to determine, for each of the transaction types, an arithmetic average of the query rate per second for the transaction type based on a response time of each of the data requests belonging to the transaction type;
and the first obtaining subunit is used for processing the response time coefficient, the arithmetic mean value of each transaction type and a preset response time index value to obtain an online processing evaluation value of each transaction type.
In another embodiment provided by the present invention, the evaluation unit 505 of the apparatus may be configured to:
the averaging processing subunit is used for carrying out averaging processing on each online processing evaluation value to obtain an online processing evaluation average value;
and the second obtaining subunit is used for processing each online processing evaluation value and the online processing evaluation average value based on a preset operation mode to obtain an online processing fluctuation range.
In another embodiment provided by the present invention, the evaluation unit 505 of the apparatus may be configured to:
the first marking subunit is used for marking the performance dimension to which the online processing fluctuation range belongs as a first target performance dimension in all preset performance dimensions, and marking all the remaining performance indexes as second target performance dimensions;
a second marking subunit, configured to mark an evaluation index corresponding to the online processing fluctuation range in the first target performance dimension, and obtain an index value of each unmarked evaluation index in the first target performance dimension;
and the generation subunit is used for generating an evaluation result of the system in the first target performance dimension based on the online processing fluctuation range and each index value.
In another embodiment provided by the present invention, the apparatus further comprises:
a fourth determining unit, configured to determine evaluation indexes of each of the second target performance dimensions;
an acquisition unit configured to acquire an index value of each evaluation index of each of the second target performance dimensions;
and the generating unit is used for generating an evaluation result of the system in the second target performance dimension based on the index values of the evaluation indexes of the second target performance dimension.
The embodiment of the invention also provides a storage medium, which comprises a stored instruction, wherein when the instruction runs, the equipment where the storage medium is located is controlled to execute the system performance evaluation method.
The electronic device according to an embodiment of the present invention has a schematic structural diagram as shown in fig. 6, and specifically includes a memory 601, and one or more instructions 602, where the one or more instructions 602 are stored in the memory 601, and are configured to be executed by one or more processors 603, and the one or more instructions 602 perform the above-mentioned method for evaluating system performance.
The specific implementation procedures and derivatives thereof of the above embodiments are within the scope of the present invention.
All the embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, the system or system embodiments are substantially similar to the method embodiments and therefore are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for related points. The above-described system and system embodiments are only illustrative, wherein the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement without inventive effort.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A method for evaluating system performance, comprising:
determining data sampling time, and acquiring each data request of a system in the data sampling time;
determining a transaction type to which each data request belongs;
determining a request proportion of each transaction type based on the number of each data request of each transaction type, and determining a response time coefficient of the system by applying each request proportion;
applying the response time coefficient to determine an online processing evaluation value of the system for each transaction type;
and calculating each online processing evaluation value to obtain an online processing fluctuation range, and evaluating the performance of the system by using the online processing fluctuation range.
2. The method of claim 1, wherein said applying each of said request ratios to determine a response time factor of said system comprises:
acquiring the complexity of each transaction type;
calculating the complexity and the request proportion of each transaction type to obtain a response coefficient of each transaction type;
and summing the response coefficients to obtain the response time coefficient of the system.
3. The method of claim 1, wherein said applying said response time factor to determine online processing estimates for said system for each transaction type comprises:
determining a response time for each data request;
for each transaction type, determining an arithmetic mean of the query rate per second for that transaction type based on the response time of each data request belonging to that transaction type;
and processing the response time coefficient, the arithmetic mean value of each transaction type and a preset response time index value to obtain an online processing evaluation value of each transaction type.
4. The method of claim 1, wherein said computing each of said online process estimates to obtain an online process fluctuation range comprises:
carrying out averaging processing on each online processing evaluation value to obtain an online processing evaluation average value;
and processing each online processing evaluation value and the online processing evaluation average value based on a preset operation mode to obtain an online processing fluctuation range.
5. The method of claim 1, wherein the using the online processing fluctuation range to evaluate the performance of the system comprises:
in the preset performance dimensions, marking the performance dimension to which the online processing fluctuation range belongs as a first target performance dimension, and marking the remaining performance indexes as second target performance dimensions;
marking the evaluation indexes corresponding to the online processing fluctuation range in the first target performance dimension, and acquiring index values of all unmarked evaluation indexes in the first target performance dimension;
and generating an evaluation result of the system in the first target performance dimension based on the online processing fluctuation range and each index value.
6. The method of claim 5, further comprising:
determining each evaluation index of each second target performance dimension;
acquiring an index value of each evaluation index of each second target performance dimension;
and for each second target performance dimension, generating an evaluation result of the system in the second target performance dimension based on the index values of the evaluation indexes of the second target performance dimension.
7. An apparatus for evaluating system performance, comprising:
the system comprises an acquisition unit, a data processing unit and a data processing unit, wherein the acquisition unit is used for determining data sampling time and acquiring each data request of the system in the data sampling time;
a first determining unit, configured to determine a transaction type to which each of the data requests belongs;
a second determining unit, configured to determine a request proportion for each transaction type based on the number of data requests for each transaction type, and determine a response time coefficient of the system by applying each request proportion;
a third determining unit, configured to apply the response time coefficient to determine an online processing evaluation value of the system for each transaction type;
and the evaluation unit is used for calculating each online processing evaluation value to obtain an online processing fluctuation range, and evaluating the performance of the system by using the online processing fluctuation range.
8. The apparatus of claim 7, wherein the second determining unit comprises:
an obtaining subunit, configured to obtain complexity of each transaction type;
the operation subunit is used for operating the complexity and the request duty ratio of each transaction type to obtain a response coefficient of each transaction type;
and the summation processing subunit is used for carrying out summation processing on each response coefficient to obtain a response time coefficient of the system.
9. A storage medium comprising stored instructions, wherein the instructions, when executed, control a device on which the storage medium resides to perform a method of evaluating system performance according to any one of claims 1-6.
10. An electronic device comprising a memory, and one or more instructions, wherein the one or more instructions are stored in the memory and configured to be executed by the one or more processors to perform the method of evaluating performance of a system according to any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210739958.7A CN114996112A (en) | 2022-06-28 | 2022-06-28 | System performance evaluation method and device, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210739958.7A CN114996112A (en) | 2022-06-28 | 2022-06-28 | System performance evaluation method and device, storage medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114996112A true CN114996112A (en) | 2022-09-02 |
Family
ID=83037786
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210739958.7A Pending CN114996112A (en) | 2022-06-28 | 2022-06-28 | System performance evaluation method and device, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114996112A (en) |
-
2022
- 2022-06-28 CN CN202210739958.7A patent/CN114996112A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Mishra et al. | Towards characterizing cloud backend workloads: insights from google compute clusters | |
| CN108446210B (en) | System performance measurement method, storage medium and server | |
| US7028301B2 (en) | System and method for automatic workload characterization | |
| CN109615129B (en) | Real estate customer transaction probability prediction method, server and computer storage medium | |
| CN107168977B (en) | Data query optimization method and device | |
| CN101430660B (en) | Pressure model analysis method based on TPS in software performance test | |
| EP1484684B1 (en) | Method and computer system for providing a cost estimate for sizing a computer system | |
| US20220269655A1 (en) | Design and implementation of data access metrics for automated physical database design | |
| CN110377519B (en) | Performance capacity test method, device and equipment of big data system and storage medium | |
| US20140214826A1 (en) | Ranking method and system | |
| US9111022B2 (en) | Simulation techniques for predicting in-memory database systems performance | |
| CN111723004A (en) | Measuring method for agile software development, measuring data output method and device | |
| CN114996112A (en) | System performance evaluation method and device, storage medium and electronic equipment | |
| CN110737673B (en) | Data processing method and system | |
| US7117218B2 (en) | System and method for expressing and calculating a relationship between measures | |
| Loboz | Cloud resource usage: extreme distributions invalidating traditional capacity planning models | |
| CN111680083B (en) | Intelligent multi-level government financial data acquisition system and data acquisition method | |
| CN112784435B (en) | GPU real-time power modeling method based on performance event counting and temperature | |
| CN110717244B (en) | Data trust analysis computer simulation method based on average deviation algorithm | |
| Ghosh | Statistical Databases for Automated Manufacturing | |
| Kang et al. | Backlog estimation and management for real-time data services | |
| Guo et al. | Green data analytics of supercomputing from massive sensor networks: Does workload distribution matter? | |
| Frolov | Benchmarking of different approaches for objects matching | |
| CN117522419B (en) | Resource allocation method applied to customer relationship management system | |
| CN116011758B (en) | Multi-data analysis intelligent integration scheduling system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |