CN115712549A - Performance evaluation method, device and storage medium - Google Patents
Performance evaluation method, device and storage medium Download PDFInfo
- Publication number
- CN115712549A CN115712549A CN202211468905.2A CN202211468905A CN115712549A CN 115712549 A CN115712549 A CN 115712549A CN 202211468905 A CN202211468905 A CN 202211468905A CN 115712549 A CN115712549 A CN 115712549A
- Authority
- CN
- China
- Prior art keywords
- physical machine
- performance
- storage performance
- pressure data
- load
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the application provides a performance evaluation method, equipment and a storage medium. In the performance evaluation method, the analysis of the performance of the storage system can be converted into the analysis of the storage performance of the physical machines, and the performance evaluation model of each physical machine is determined according to the actual operation data of the physical machine, so that the analysis result is closer to the actual operation condition of the physical machine, and a more accurate performance estimation result is obtained. The load pressure data are subjected to online statistics in the operation process of the storage system and can be used for reflecting the real pressure condition of the storage system, so that the predicted storage performance is more real and reliable, the risk of bringing extra load to a physical machine is reduced, and the influence on the service capacity of the storage system is reduced.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a performance evaluation method, device, and storage medium.
Background
When providing storage services, a storage system should increase the utilization rate of storage resources as much as possible to optimize the cost. Therefore, accurately evaluating the performance of the storage system under the condition of a given storage resource is very critical for guaranteeing the quality of the storage service and improving the resource utilization rate of the storage system.
However, the performance of the storage system cannot be estimated more accurately by the conventional performance estimation method. Therefore, a solution is yet to be proposed.
Disclosure of Invention
Aspects of the present disclosure provide a performance evaluation method, device, and storage medium for evaluating performance of a storage system more accurately.
The embodiment of the application provides a performance evaluation method, which is suitable for a storage system, wherein the storage system comprises a plurality of physical machines; the method comprises the following steps: in the running process of a storage system, carrying out statistics on access operations received by a plurality of physical machines in the storage system to obtain load pressure data of the physical machines; inputting the load pressure data of the plurality of physical machines into respective performance evaluation models of the plurality of physical machines to obtain the estimated storage performance of the plurality of physical machines; determining a performance evaluation model of any one of the plurality of physical machines according to historical load pressure data of the physical machine and corresponding historical storage performance; and determining the storage performance of the storage system according to the estimated storage performance of the plurality of physical machines.
Optionally, before inputting the load pressure data of the plurality of physical machines into the performance evaluation model of each of the plurality of physical machines, the method further includes: carrying out pressure test on the physical machine according to a preset load pressure data sample; in the pressure testing process, acquiring equipment operating parameters of the physical machine; determining the load state of the physical machine according to the equipment operation parameters of the physical machine; when the load state of the physical machine is a balanced state, acquiring the actual storage performance of the physical machine as a storage performance sample; and fitting the performance change trend corresponding to the physical machine according to the load pressure data sample and the storage performance sample to obtain a performance evaluation model corresponding to the physical machine.
Optionally, the load pressure data of any physical machine includes: at least one of a concurrency of disk read operations, an I/O size, a read-write I/O ratio, and a ratio of sequential access data.
Optionally, after obtaining the estimated storage performances of the plurality of physical machines, the method further includes: and correcting the performance evaluation models corresponding to the physical machines according to the load pressure data of the physical machines and the estimated storage performance.
Optionally, modifying the performance evaluation model corresponding to each of the plurality of physical machines according to the load pressure data of the plurality of physical machines and the estimated storage performance includes: for a target physical machine in the plurality of physical machines, acquiring equipment operating parameters of the target physical machine and actual storage performance provided by the target physical machine; determining the load state of the target physical machine according to the equipment operation parameters of the target physical machine; updating a mapping table of load pressure data and storage performance according to the load state, the actual storage performance and the estimated storage performance of the target physical machine; and correcting the performance evaluation model of the target physical machine according to the mapping table according to a set updating period.
Optionally, updating the mapping table of the load pressure data and the storage performance according to the load state, the actual storage performance, and the estimated storage performance of the target physical machine, including: if the actual storage performance is larger than a first threshold value, judging whether the load state of the target physical machine is a light load state or a balanced state; the first threshold value is determined according to the estimated storage performance and an error upper limit value; and if the load state of the target physical machine is a light load state or a balanced state, updating the mapping table according to the corresponding relation between the load pressure data of the target physical machine and the actual storage performance.
Optionally, updating a mapping table of load pressure data and storage performance according to the load state, actual storage performance, and estimated storage performance of the target physical machine, including: if the actual storage performance is smaller than a second threshold value, judging whether the load state of the target physical machine is a light load state, an overload state or a balance state; the second threshold value is determined according to the estimated storage performance and the error lower limit value; if the load state of the target physical machine is a light load state or a balanced state overload state, updating the mapping table according to the corresponding relation between the load pressure data of the target physical machine and the actual storage performance; and if the load state of the target physical machine is an overload state, correcting the actual storage performance of the target physical machine according to a preset convergence step length, and updating the mapping table according to the corresponding relation between the corrected actual storage performance and the load pressure data of the target physical machine.
Optionally, updating a mapping table of load pressure data and storage performance according to the load state, actual storage performance, and estimated storage performance of the target physical machine, including: if the actual storage performance is larger than or equal to a second threshold value and smaller than or equal to a first threshold value, judging whether the load state of the target physical machine is an overload state; the first threshold value is determined according to the estimated storage performance and an error upper limit value; the second threshold is determined according to the estimated storage performance and the error lower limit value; and if the load state of the target physical machine is an overload state, correcting the actual storage performance of the target physical machine according to a preset convergence step length, and updating the mapping table according to the corresponding relation between the corrected actual storage performance and the load pressure data of the target physical machine.
An embodiment of the present application further provides a server, including: a memory and a processor; the memory is to store one or more computer instructions; the processor is to execute the one or more computer instructions to: the steps in the method provided by the embodiments of the present application are performed.
Embodiments of the present application further provide a computer-readable storage medium storing a computer program, where the computer program can implement the steps in the method provided in the embodiments of the present application when executed by a processor.
In the embodiment of the application, in the operation process of the storage system, the load pressure data of the physical machine in the storage system can be acquired in a statistical mode, and the estimated storage performance of the physical machine is acquired based on a predetermined performance evaluation model. Based on the estimated storage performance of the plurality of physical machines, the overall storage performance of the storage system may be determined. In the embodiment, the analysis of the performance of the storage system can be converted into the analysis of the storage performance of the physical machines, and the performance evaluation model of each physical machine is determined according to the actual operation data of the physical machine, so that the analysis result is closer to the actual operation condition of the physical machine, and a more accurate performance estimation result is obtained. The load pressure data is subjected to online statistics in the operation process of the storage system and can be used for reflecting the real pressure condition of the storage system, so that the predicted storage performance is more real and reliable, the risk of bringing extra load to a physical machine is reduced, and the influence on the service capacity of the storage system is reduced.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
FIG. 1 is a schematic flow chart diagram of a performance evaluation method provided in an exemplary embodiment of the present application;
FIG. 2 is a schematic diagram of a performance assessment tool provided in an exemplary embodiment of the present application;
fig. 3 illustrates a schematic structural diagram of a server according to an exemplary embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terminology used in the embodiments of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the examples of the invention and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise, and "the plural" typically includes at least two, but does not exclude the presence of at least one.
It should be understood that the term "and/or" as used herein is merely one type of association that describes an associated object, meaning that three relationships may exist, e.g., a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter associated objects are in an "or" relationship.
It is also noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a good or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such good or system. Without further limitation, an element defined by the phrases "comprising one of 8230; \8230;" 8230; "does not exclude the presence of additional like elements in articles of commerce or systems including such elements.
In some embodiments of the present application, a solution is provided, and the following describes in detail the technical solutions provided by the embodiments of the present application with reference to the accompanying drawings.
Fig. 1 is a schematic flowchart of a performance evaluation method provided in an exemplary embodiment of the present application, where the method is applied to a storage system, and the storage system includes a plurality of physical machines. As shown in fig. 1, the method may include:
102, inputting the load pressure data of the physical machines into respective performance evaluation models of the physical machines to obtain the estimated storage performance of the physical machines; the performance evaluation model of any one of the plurality of physical machines is determined according to historical load pressure data of the physical machine and corresponding historical storage performance.
And 103, determining the storage performance of the storage system according to the estimated storage performance of the plurality of physical machines.
The execution subject of the embodiment may be a performance evaluation tool, which includes an online data monitoring module and a performance evaluation module. The online data monitoring module can be respectively deployed on each physical machine in the storage system, and monitors load pressure data of the physical machines in the operation process on line. The performance evaluation module can run on the background process of each physical machine in a light weight mode, so that the influence on foreground storage service of the physical machine is reduced.
Wherein the load pressure data is used for describing the access pressure received by the physical machine when the storage service is provided. The load pressure data may include one or more metrics describing the load pressure of the stand-alone storage modules. In some alternative embodiments, the load pressure data may include: at least one of concurrency of disk read operations, I/O (Input/Output) size (i.e., I/Osize), read-write I/O ratio (RWRatio), and ratio of sequential access data (Randomness). The proportion of the sequential access data is used for describing the operation of accessing the data from the disk of the physical machine according to the specified sequence, and the proportion of all the access operations received on the disk of the single memory device in the set period.
In some optional embodiments, during the operation of the storage system, the online data monitoring module may count access operations received by each of a plurality of physical machines in the storage system, so as to obtain load pressure data of the plurality of physical machines. The access operation for the physical machine is typically a disk read operation. Based on the embodiment, in the performance evaluation process, only the statistics of the disk reading behavior when the storage system provides the storage service is needed, and the disk reading operation is not needed to be additionally introduced, so that the additional I/O pressure added to the single-machine storage system is avoided. Meanwhile, in the implementation mode, the load pressure data is acquired online in the operation process of the storage system and can be used for reflecting the real pressure condition of the storage system, so that the predicted storage performance is more real and reliable.
After the online data monitoring module on any physical machine acquires the load pressure data of the physical machine, the load pressure data can be sent to the performance evaluation module corresponding to the physical machine. The performance evaluation module can input the load pressure data of the physical machine into the performance evaluation model corresponding to the physical machine to obtain the estimated storage performance of the physical machine.
The storage performance of any physical machine can be described by adopting a preset performance index. In some embodiments, the performance indicators may include, but are not limited to: IOPS (Input/output operation Perform, number of reads and writes per second), access latency, throughput rate, and throughput bandwidth. Generally, the smaller the IOPS of the physical machine, the smaller the access delay, the higher the throughput rate, or the larger the throughput bandwidth, the better the performance of the physical machine.
Wherein the performance evaluation models of different physical machines are different. A performance evaluation model for any physical machine is determined based on historical load pressure data for that physical machine and corresponding historical storage performance. The historical load pressure data may include pressure applied to the physical machine during a pressure test process of the physical machine, and may also include actual load pressure received when the physical machine provides storage service within a historical period of time, which is not limited in this embodiment.
The historical storage performance refers to the actual storage performance of the physical machine under the historical load pressure. The historical load pressure data and the corresponding historical storage performance can reflect the real running condition of the physical machine. The performance evaluation model can learn the real performance change of the physical machine under different load pressures based on historical load pressure data and corresponding historical storage performance, so that the storage performance of the physical machine can be predicted more accurately.
After the respective estimated storage performance of the plurality of physical machines is determined, the storage performance of the storage system can be determined according to the respective estimated storage performance of the plurality of physical machines. The storage performance of the storage system may be an average value or a weighted average value of the estimated storage performances of the plurality of physical machines, which is not limited in this embodiment.
In this embodiment, during the operation of the storage system, load pressure data of the physical machine in the storage system may be obtained in a statistical manner, and the estimated storage performance of the physical machine may be obtained based on a predetermined performance evaluation model. Based on the estimated storage performance of the plurality of physical machines, the overall storage performance of the storage system may be determined. In the embodiment, the analysis on the performance of the storage system can be converted into the analysis on the storage performance of the physical machines, and the performance evaluation model of each physical machine is determined according to the actual operation data of the physical machine, so that the analysis result is closer to the actual operation condition of the physical machine, and a more accurate performance estimation result is obtained. The load pressure data is subjected to online statistics in the operation process of the storage system and can be used for reflecting the real pressure condition of the storage system, so that the predicted storage performance is more real and reliable, the risk of bringing extra load to a physical machine is reduced, and the influence on the service capacity of the storage system is reduced.
In some optional embodiments, before the load pressure data of the plurality of physical machines is input into the performance evaluation model of each of the plurality of physical machines, the actual performance data of the physical machines in the storage system may be obtained by testing the storage system, and the performance evaluation model of the physical machines may be established according to the actual performance data.
When initializing performance evaluation models of different physical machines, the physical machines with the same machine type can have the same performance model; the initialized performance models of the physical machines of different models may be the same or different. In the fitting process, each physical machine can independently fit a performance evaluation model of the physical machine, and further, even if the performance evaluation models during initialization are the same, due to different running conditions and environments of different storage devices, the performance evaluation models matched with the actual running conditions of each physical machine can be obtained through fitting.
The following description will be exemplified by taking any one of the physical machines in the storage system as an example.
Optionally, for any physical machine, the physical machine may be subjected to a pressure test according to a preset load pressure data sample. And acquiring equipment operating parameters of the physical machine in the pressure test process.
The device operation parameters comprise software operation indexes and/or hardware operation indexes used for reflecting whether the single storage device reaches the performance bottleneck. The device operation parameters may be obtained by the data monitoring module from components such as a Central Processing Unit (CPU), a memory, and a disk, as shown in fig. 2. Obtaining device operating parameters may include, but is not limited to: at least one of a disk busy level index (distutil), a disk processing queue, a disk I/O processing delay, a cache water level, a CPU utilization rate and a memory utilization rate.
And determining the load state of the physical machine according to the equipment operation parameters of the physical machine. The load state of the physical machine may include: overload state, equilibrium state, or light load state.
The load state of the physical machine can be determined according to the load states of various equipment operation parameters. For example, each plant operating parameter may be divided into three ranges of values, which may correspond to an equalization state, a light load state, and an overload state, respectively. For any equipment operation parameter, the load state corresponding to the equipment operation parameter can be determined according to the value interval range of the index value. For example, taking the implementation of the device operation parameter as the CPU utilization, assuming that the range of the numerical interval corresponding to the light-load state of the CPU utilization is [0,50% ], the range of the numerical interval corresponding to the equilibrium state is (50%, 80% ], and the range of the numerical interval corresponding to the overload state is (80%, 100% ], if the currently monitored CPU utilization of the physical machine is 65%, it is determined that the load state corresponding to the CPU utilization is the equilibrium state.
Optionally, if at least one index in the device operation parameters of the physical machine is in an overload state, the physical machine may be considered to be in the overload state. Optionally, if all the indexes in the device operation parameters of the physical machine are in a balanced state, the physical machine may be considered to be in a balanced state. Alternatively, if there is no indicator in the overload state but there is at least one indicator in the light load state in the device operation parameters of the physical machine, it may be determined that the physical machine is in the light load state.
And when the load state of the physical machine is a balanced state, acquiring the actual storage performance of the physical machine as the storage performance sample. The actual storage performance may be a storage performance peak of the physical machine in a load balancing state. After the load pressure data sample and the storage performance sample are obtained, the performance change trend corresponding to the physical machine can be fitted according to the load pressure data sample and the storage performance sample, and the performance evaluation model corresponding to the physical machine is obtained. Wherein, the evaluation process of the performance evaluation model can be expressed by the following formula:
Perf e ＝Func(X1，X2…，Xn)
wherein, perf e Representing the estimated storage performance, xn representing the nth load pressure data, func () representing the performance evaluation model.
Multiple sets of pressure testing procedures can be performed. In the process of testing multiple groups of samples, parameter values in the load pressure data samples can be adjusted to test the storage performance of the physical machine under different pressures, so that multiple groups of sample data are obtained. Correspondingly, the performance variation trend can be obtained based on the fitting of multiple groups of sample data, that is, the model parameters in Func () can be dynamically adjusted according to the actual pressure test process, so that the performance evaluation model can adapt to the performance prediction requirements under different load pressures.
The performance variation trend may be expressed by a performance curve, and the fitting manner of the performance curve may include a least square method, a method of approximating discrete data by an analytic expression, or a method based on a neural network model, which is not limited in this embodiment.
In the embodiment, when the performance evaluation model is established, the load state of the physical machine is monitored by the equipment operation parameters of the physical machine, so that the performance evaluation model and the load state of the physical machine can be associated, and the influence of objective performance reduction caused by hardware wear and aging of the physical machine on the storage performance estimation result is reduced.
In some optional embodiments, after obtaining the estimated storage performance of the physical machine during the operation of the storage system, the currently used performance evaluation model may be further modified to optimize the performance evaluation model. That is, the performance evaluation model of each of the plurality of physical machines may be modified based on the load pressure data of each of the plurality of physical machines and the estimated storage performance. The load pressure data of each physical machine acquired online and the corresponding estimated storage performance output by the performance evaluation model can be stored in a designated file, so that the performance evaluation module corrects the performance evaluation model corresponding to each physical machine according to a set updating period.
The following description will be exemplified by taking any one target physical machine among the plurality of physical machines as an example.
Optionally, the online data monitoring module may obtain the device operating parameters of the target physical machine and the actual storage performance provided by the target physical machine. The actual storage performance refers to the storage performance that the physical machine can actually provide under the load pressure corresponding to the load pressure data. Wherein, according to the equipment operating parameters of the target physical machine, the data monitoring module can determine the load state of the target physical machine. For an optional implementation of determining the load status according to the device operation parameters, reference may be made to the descriptions of the foregoing embodiments, which are not repeated herein.
After the data monitoring module acquires the load state and the actual storage performance of the target physical machine, the load state and the actual storage performance of the target physical machine can be sent to the performance evaluation module according to a set sending period. The performance evaluation module can update the mapping table of the load pressure data and the storage performance according to the load state, the actual storage performance and the corresponding estimated storage performance of the target physical machine. The mapping table may be updated one or more times during an update period. The performance evaluation module can modify the performance evaluation model of the target physical machine according to the set updating period and the mapping table. In this embodiment, the performance evaluation model is continuously corrected by the online data, so that the accuracy and reliability of the performance evaluation model can be further improved.
In the above embodiments, when the load state of the physical machine is different, the updating manner of the mapping table of the load pressure data and the storage performance is different, and will be specifically described below.
Alternatively, the performance evaluation module may determine a relationship of the actual storage performance of the physical machine to the first threshold and the second threshold. The first threshold is determined according to the estimated storage performance and the error upper limit value. The second threshold is determined according to the estimated storage performance and the error lower limit value.
Alternatively, assuming that the error upper limit value is a and the error lower limit value is-a, the first threshold value may be expressed as:
Perf e *(1+a)
the second threshold may be expressed as:
Perf e *(1-a)
wherein, perf e Representing the actual predicted storage performance. Wherein, 0<α<1, for example, in some embodiments, α =0.05 may be set.
In some alternative embodiments, if the actual storage performance of the physical machine is Perf r Greater than the first threshold, then it may be considered to beThe actual storage performance of the physical machine is larger than the estimated storage performance, namely, the estimated result of the performance evaluation model is lower.
In this case, the performance evaluation module may determine whether the load state of the target physical machine is a light load state or a balanced state; if the load state of the target physical machine is a light load state or a balanced state, the performance evaluation module can update the mapping table according to the corresponding relation between the load pressure data of the target physical machine and the actual storage performance. If the load state of the target physical machine is an overload state, the fact that the actual storage performance is larger than the estimated storage performance is caused by a large load, and at the moment, the corresponding relation between the load pressure data and the actual storage performance does not need to be updated into the mapping table.
I.e. if Perf e *(1+a)<Perf r And if the load state of the physical machine is light load or balanced, the performance estimated by the performance evaluation model is not accurate, and the performance evaluation model needs to be corrected. At this time, the corresponding relationship between the load pressure data and the actual storage performance of the physical machine under the load pressure data, namely [ X1, X2 \8230 ], xn]->Perf r [X1，X2…，Xn]And updating the mapping table of the performance evaluation model of the physical machine.
In other alternative embodiments, if the actual storage performance of the physical machine is less than the second threshold, the actual storage performance of the physical machine may be considered less than the estimated storage performance. That is, the estimation result of the performance evaluation model is high.
In this case, the performance evaluation module may determine whether the load status of the target physical machine is a light load status, an overload status, or a balanced status. And if the load state of the target physical machine is a light load state or a balanced state, the mapping table can be updated according to the corresponding relation between the load pressure data of the target physical machine and the actual storage performance. And if the load state of the target physical machine is an overload state, correcting the actual storage performance of the target physical machine according to a preset convergence step length, and updating the mapping table according to the corresponding relation between the corrected actual storage performance and the load pressure data of the target physical machine.
The convergence step length is used for reducing the actual storage performance of the target physical machine in the overload state, so that the actual storage performance of the target physical machine in the overload state is closer to the actual storage performance of the target physical machine in the balanced state or the light load state. In the process of correcting the performance storage model of the target physical machine according to the mapping table, the performance model of the target physical machine can continuously learn the performance change trend of the target physical machine in a balanced state or a light-load state. Furthermore, in the subsequent prediction process, the target physical machine can predict the performance of the target physical machine in a load balancing state or a light load state according to the load pressure data acquired in real time. The prediction result can be used for performing access control (or flow control) on the target physical machine to adjust the load pressure of the target physical machine, so that the influence on the storage service of the target physical machine is reduced, and the service stability of the target physical machine is improved.
I.e. if Perf e *(1-a)>Perf r And the load state of the target physical machine is a light load state or a balanced state, and the performance estimated by the performance evaluation model is not accurate, so that the performance evaluation model needs to be corrected. The performance evaluation module can correspond the load pressure data to the actual storage performance of the physical machine under the load pressure data, namely [ X1, X2 \8230 ], xn]->Perf r [X1，X2…，Xn]And updating the mapping table of the performance evaluation model of the physical machine.
If the load state of the target physical machine is an overload state, the performance estimated by the performance evaluation model is not accurate, and the performance evaluation model needs to be corrected. The performance evaluation module can correct the actual storage performance of the target physical machine under the load pressure data according to the convergence step length to obtain (1-beta) × Perf r [X1，X2…，Xn]. The corresponding relation between the load pressure data and the actual storage performance after correction, namely [ X1, X2 \8230 ], xn]->(1-β)*Perf r [X1，X2…，Xn]And updating the mapping table of the performance evaluation model of the physical machine. Wherein, 0<β<And 1, the convergence step length is used for representing the convergence step length of the correction performance evaluation model, and the convergence step length can be selected according to the convergence speed of the performance evaluation model.The larger β is, the faster the convergence speed of the performance evaluation model is. In some embodiments, β =0.05 may be desirable to enable a more stable convergence of the performance assessment model to the target state. The performance evaluation model after converging to the target state can enable the predicted storage performance to better accord with the current objectively existing equipment conditions of the target physical machine, so that the risk of overload operation of the target physical machine is reduced.
In some alternative embodiments, the estimate of the performance evaluation model may be considered reasonable if the actual storage performance is greater than or equal to the second threshold and less than or equal to the first threshold. In this case, the performance evaluation model may be further optimized in combination with the load status of the target physical machine. Optionally, the performance evaluation module may determine whether the load status of the target physical machine is an overload status. If the load state of the target physical machine is an overload state, the target physical machine can be considered to have higher load pressure when reaching the actual performance, at this time, the actual storage performance of the target physical machine can be corrected according to the preset convergence step length, and the mapping table is updated according to the corresponding relation between the corrected actual storage performance and the load pressure data of the target physical machine. If the load state of the target physical machine is a light load state or a balanced state, the corresponding relation between the load pressure data and the actual storage performance does not need to be updated into the mapping table.
That is, if Perf e *(1-a)≤Perf r ≤Perf e * (1 + a), and the load state of the target physical machine is an overload state, the performance estimated by the performance estimation model is not accurate, and the performance estimation model needs to be corrected. The performance evaluation module can correct the actual storage performance of the target physical machine under the load pressure data according to the convergence step length to obtain (1-beta) × Perf r [X1，X2…，Xn]. The corresponding relation between the load pressure data and the actual storage performance after correction, namely [ X1, X2 \8230 ], xn]->(1-β)*Perf r [X1，X2…，Xn]And updating the mapping table of the performance evaluation model of the physical machine. By setting the value of β, it is possible to enable the performance evaluation model to stably converge to the target state. Performance assessment after convergence to target stateThe model can enable the predicted storage performance to better accord with the current objectively existing equipment conditions of the target physical machine, so that the risk of overload operation of the target physical machine is reduced.
In the embodiment, the performance evaluation model of the physical machine is corrected by combining the load state of the physical machine, and the performance reduction of the physical machine caused by objective reasons such as hardware wear and aging can be used as an implicit variable of the performance evaluation model, so that the performance evaluation model is more adaptive to the actual running condition of the physical machine, and a more accurate storage performance estimation result is obtained.
It should be noted that, the executing subjects of the steps of the method provided in the foregoing embodiments may be the same device, or different devices may also be used as the executing subjects of the method. For example, the execution subject of steps 101 to 104 may be device a; for another example, the execution subject of steps 101 and 102 may be device a, and the execution subject of step 103 may be device B; and so on.
In addition, in some of the flows described in the above embodiments and the drawings, a plurality of operations occurring in a specific order are included, but it should be clearly understood that these operations may be executed out of order or in parallel as they appear herein, and the sequence numbers of the operations, such as 101, 102, etc., are used merely to distinguish various operations, and the sequence numbers themselves do not represent any execution order. Additionally, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first", "second", etc. in this document are used for distinguishing different messages, devices, modules, etc., and do not represent a sequential order, nor limit the types of "first" and "second" to be different.
Fig. 3 illustrates a schematic structural diagram of a server provided in an exemplary embodiment of the present application, where the server may be implemented as any physical machine in a storage system, and the storage system includes a plurality of physical machines. As shown in fig. 3, the server includes: memory 301, processor 302, and communication component 303.
The memory 301 is used for storing computer programs and may be configured to store other various data to support operations on the server. Examples of such data include instructions for any application or method operating on the server.
A processor 302, coupled to the memory 301, for executing the computer program in the memory 301 to: in the running process of a storage system, carrying out statistics on access operations received by a plurality of physical machines in the storage system to obtain load pressure data of the physical machines; inputting the load pressure data of the plurality of physical machines into respective performance evaluation models of the plurality of physical machines to obtain the estimated storage performance of the plurality of physical machines; determining a performance evaluation model of any one of the plurality of physical machines according to historical load pressure data of the physical machine and corresponding historical storage performance; and determining the storage performance of the storage system according to the estimated storage performances of the plurality of physical machines.
Optionally, the processor 302, before inputting the load pressure data of the plurality of physical machines into the performance evaluation model of each of the plurality of physical machines, is further configured to: carrying out pressure test on the physical machine according to a preset load pressure data sample; in the pressure testing process, acquiring equipment operating parameters of the physical machine; determining the load state of the physical machine according to the equipment operation parameters of the physical machine; when the load state of the physical machine is a balanced state, acquiring the actual storage performance of the physical machine as a storage performance sample; and fitting the performance change trend corresponding to the physical machine according to the load pressure data sample and the storage performance sample to obtain a performance evaluation model corresponding to the physical machine.
Optionally, the load pressure data of any physical machine includes: at least one of a concurrency of disk read operations, an I/O size, a read-write I/O ratio, and a ratio of sequential access data.
Optionally, after obtaining the estimated storage performance of the plurality of physical machines, the processor 302 is further configured to: and correcting the performance evaluation models corresponding to the physical machines according to the load pressure data of the physical machines and the estimated storage performance.
Optionally, the modifying, by the processor 302, the performance evaluation model corresponding to each of the plurality of physical machines according to the load pressure data of the plurality of physical machines and the estimated storage performance includes: aiming at a target physical machine in the plurality of physical machines, acquiring equipment operating parameters of the target physical machine and actual storage performance provided by the target physical machine; determining the load state of the target physical machine according to the equipment operation parameters of the target physical machine; updating a mapping table of load pressure data and storage performance according to the load state, the actual storage performance and the estimated storage performance of the target physical machine; and correcting the performance evaluation model of the target physical machine according to the mapping table according to the set updating period.
Optionally, when the mapping table of the load pressure data and the storage performance is updated according to the load state, the actual storage performance, and the estimated storage performance of the target physical machine, the processor 302 is specifically configured to: if the actual storage performance is larger than a first threshold value, judging whether the load state of the target physical machine is a light load state or a balanced state; the first threshold value is determined according to the estimated storage performance and an error upper limit value; and if the load state of the target physical machine is a light load state or a balanced state, updating the mapping table according to the corresponding relation between the load pressure data of the target physical machine and the actual storage performance.
Optionally, when the mapping table of load pressure data and storage performance is updated according to the load state, the actual storage performance, and the estimated storage performance of the target physical machine, the processor 302 is specifically configured to: if the actual storage performance is smaller than a second threshold value, judging whether the load state of the target physical machine is a light load state, an overload state or a balance state; the second threshold value is determined according to the estimated storage performance and the error lower limit value; if the load state of the target physical machine is a light load state or a balanced state overload state, the mapping table is updated according to the corresponding relation between the load pressure data of the target physical machine and the actual storage performance; and if the load state of the target physical machine is an overload state, correcting the actual storage performance of the target physical machine according to a preset convergence step length, and updating the mapping table according to the corresponding relation between the corrected actual storage performance and the load pressure data of the target physical machine.
Optionally, when the mapping table of the load pressure data and the storage performance is updated according to the load state, the actual storage performance, and the estimated storage performance of the target physical machine, the processor 302 is specifically configured to: if the actual storage performance is greater than or equal to a second threshold value and less than or equal to a first threshold value, judging whether the load state of the target physical machine is an overload state; wherein, the first threshold value is determined according to the estimated storage performance and the error upper limit value; the second threshold value is determined according to the estimated storage performance and the error lower limit value; and if the load state of the target physical machine is an overload state, correcting the actual storage performance of the target physical machine according to a preset convergence step length, and updating the mapping table according to the corresponding relation between the corrected actual storage performance and the load pressure data of the target physical machine.
Further, as shown in fig. 3, the server further includes: power supply component 304, and the like. Only some of the components are schematically shown in fig. 3, and it is not meant that the server includes only the components shown in fig. 3.
The memory 301 may be implemented, among other things, by any type of volatile or non-volatile storage device or combination thereof, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.
Wherein the communication component 303 is configured to facilitate communication between the device in which the communication component is located and other devices in a wired or wireless manner. The device in which the communication component is located may access a wireless network based on a communication standard, such as WiFi,2G, 3G, 4G, or 5G, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component may be implemented based on Near Field Communication (NFC) technology, radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.
The power component 304 is used to provide power to various components of the device in which the power component is located. The power components may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device in which the power component is located.
In this embodiment, in the process of operating the storage system, load pressure data of the physical machine in the storage system may be obtained in a statistical manner, and the estimated storage performance of the physical machine may be obtained based on a predetermined performance evaluation model. Based on the estimated storage performance of the plurality of physical machines, an overall storage performance of the storage system may be determined. In the embodiment, the analysis on the performance of the storage system can be converted into the analysis on the storage performance of the physical machines, and the performance evaluation model of each physical machine is determined according to the actual operation data of the physical machine, so that the analysis result is closer to the actual operation condition of the physical machine, and a more accurate performance estimation result is obtained. The load pressure data is subjected to online statistics in the operation process of the storage system and can be used for reflecting the real pressure condition of the storage system, so that the predicted storage performance is more real and reliable, the risk of bringing extra load to a physical machine is reduced, and the influence on the service capacity of the storage system is reduced.
Accordingly, the present application further provides a computer-readable storage medium storing a computer program, where the computer program can implement the steps that can be executed by the server in the foregoing method embodiments when executed.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Disks (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of other like elements in a process, method, article, or apparatus comprising the element.
The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art to which the present application pertains. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims (10)
1. A performance evaluation method is applicable to a storage system, wherein the storage system comprises a plurality of physical machines; the method comprises the following steps:
in the running process of a storage system, carrying out statistics on access operations received by a plurality of physical machines in the storage system to obtain load pressure data of the physical machines;
inputting the load pressure data of the plurality of physical machines into respective performance evaluation models of the plurality of physical machines to obtain respective estimated storage performance of the plurality of physical machines; determining a performance evaluation model of any one of the plurality of physical machines according to historical load pressure data of the physical machine and corresponding historical storage performance;
and determining the storage performance of the storage system according to the estimated storage performance of the plurality of physical machines.
2. The method of claim 1, prior to inputting the load pressure data for the plurality of physical machines into the performance assessment model for each of the plurality of physical machines, further comprising:
carrying out pressure test on the physical machine according to a preset load pressure data sample;
in the pressure testing process, acquiring equipment operating parameters of the physical machine;
determining the load state of the physical machine according to the equipment operation parameters of the physical machine;
when the load state of the physical machine is a balanced state, acquiring the actual storage performance of the physical machine as a storage performance sample;
and fitting the performance change trend corresponding to the physical machine according to the load pressure data sample and the storage performance sample to obtain a performance evaluation model corresponding to the physical machine.
3. The method of claim 1, the load pressure data for any physical machine comprising: at least one of concurrency of disk read operations, I/O size, read-write I/O ratio, and ratio of sequential access data.
4. The method of claim 1, after obtaining the estimated storage performance of the plurality of physical machines, further comprising:
and correcting the performance evaluation models corresponding to the physical machines according to the load pressure data and the estimated storage performance of the physical machines.
5. The method of claim 4, wherein modifying the performance assessment model for each of the plurality of physical machines based on the load pressure data and the estimated storage performance of the plurality of physical machines comprises:
for a target physical machine in the plurality of physical machines, acquiring equipment operating parameters of the target physical machine and actual storage performance provided by the target physical machine;
determining the load state of the target physical machine according to the equipment operation parameters of the target physical machine;
updating a mapping table of load pressure data and storage performance according to the load state, the actual storage performance and the estimated storage performance of the target physical machine;
and correcting the performance evaluation model of the target physical machine according to the mapping table according to a set updating period.
6. The method of claim 5, updating a mapping table of load pressure data to storage performance based on the load status, actual storage performance, and projected storage performance of the target physical machine, comprising:
if the actual storage performance is larger than a first threshold value, judging whether the load state of the target physical machine is a light load state or a balanced state; the first threshold value is determined according to the estimated storage performance and an error upper limit value;
and if the load state of the target physical machine is a light load state or a balanced state, updating the mapping table according to the corresponding relation between the load pressure data of the target physical machine and the actual storage performance.
7. The method of claim 5, updating a mapping table of load pressure data to storage performance based on the load status, actual storage performance, and projected storage performance of the target physical machine, comprising:
if the actual storage performance is smaller than a second threshold value, judging whether the load state of the target physical machine is a light load state, an overload state or a balance state; the second threshold is determined according to the estimated storage performance and the error lower limit value;
if the load state of the target physical machine is a light load state or a balanced state overload state, updating the mapping table according to the corresponding relation between the load pressure data of the target physical machine and the actual storage performance;
and if the load state of the target physical machine is an overload state, correcting the actual storage performance of the target physical machine according to a preset convergence step length, and updating the mapping table according to the corresponding relation between the corrected actual storage performance and the load pressure data of the target physical machine.
8. The method of claim 5, updating a mapping table of load pressure data to storage performance based on the load state, actual storage performance, and estimated storage performance of the target physical machine, comprising:
if the actual storage performance is larger than or equal to a second threshold value and smaller than or equal to a first threshold value, judging whether the load state of the target physical machine is an overload state; the first threshold value is determined according to the estimated storage performance and an error upper limit value; the second threshold is determined according to the estimated storage performance and the error lower limit value;
and if the load state of the target physical machine is an overload state, correcting the actual storage performance of the target physical machine according to a preset convergence step length, and updating the mapping table according to the corresponding relation between the corrected actual storage performance and the load pressure data of the target physical machine.
9. A server, comprising: a memory and a processor;
the memory is to store one or more computer instructions;
the processor is to execute the one or more computer instructions to: performing the steps of the method of any one of claims 1-8.
10. A computer-readable storage medium storing a computer program which, when executed by a processor, is capable of implementing the performance evaluation method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211468905.2A CN115712549A (en) | 2022-11-22 | 2022-11-22 | Performance evaluation method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211468905.2A CN115712549A (en) | 2022-11-22 | 2022-11-22 | Performance evaluation method, device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115712549A true CN115712549A (en) | 2023-02-24 |
Family
ID=85234217
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211468905.2A Pending CN115712549A (en) | 2022-11-22 | 2022-11-22 | Performance evaluation method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115712549A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116991636A (en) * | 2023-09-26 | 2023-11-03 | 武汉吧哒科技股份有限公司 | Data incremental backup method, system and storage medium based on distributed storage |
| CN117370034A (en) * | 2023-12-07 | 2024-01-09 | 之江实验室 | Evaluation method and device of computing power dispatching system, storage medium and electronic equipment |
-
2022
- 2022-11-22 CN CN202211468905.2A patent/CN115712549A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116991636A (en) * | 2023-09-26 | 2023-11-03 | 武汉吧哒科技股份有限公司 | Data incremental backup method, system and storage medium based on distributed storage |
| CN116991636B (en) * | 2023-09-26 | 2024-01-19 | 武汉吧哒科技股份有限公司 | Data incremental backup method, system and storage medium based on distributed storage |
| CN117370034A (en) * | 2023-12-07 | 2024-01-09 | 之江实验室 | Evaluation method and device of computing power dispatching system, storage medium and electronic equipment |
| CN117370034B (en) * | 2023-12-07 | 2024-02-27 | 之江实验室 | Evaluation method and device of computing power dispatching system, storage medium and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108376112B (en) | Pressure testing method, device and readable medium | |
| CN115712549A (en) | Performance evaluation method, device and storage medium | |
| US9430288B2 (en) | Job scheduling based on historical job data | |
| US8494806B2 (en) | Method, program and apparatus for optimizing configuration parameter set of system | |
| CN110572297B (en) | Network performance evaluation method, server and storage medium | |
| US11042410B2 (en) | Resource management of resource-controlled system | |
| CN112685170A (en) | Dynamic optimization of backup strategies | |
| CN110633194B (en) | Performance evaluation method of hardware resources in specific environment | |
| CN111881023B (en) | Software aging prediction method and device based on multi-model comparison | |
| GB2599062A (en) | Container-based applications | |
| CN110532154B (en) | Application system capacity expansion method, device and equipment | |
| CN111723018A (en) | Performance pressure testing method, device, equipment and storage medium | |
| CN111026553A (en) | Resource scheduling method for offline mixed part operation and server system | |
| CN106503841B (en) | method and device for determining index threshold | |
| CN103713935A (en) | Method and device for managing Hadoop cluster resources in online manner | |
| CN111881058B (en) | Software engineering quality prediction method | |
| CN111897706A (en) | Server performance prediction method, device, computer system and medium | |
| CN108897673B (en) | System capacity evaluation method and device | |
| US20220398021A1 (en) | Workload management using a trained model | |
| CN112398670A (en) | Flow prediction method, flow prediction device, computing equipment and computer storage medium | |
| US9501321B1 (en) | Weighted service requests throttling | |
| CN117573347A (en) | Resource allocation method and system of cloud computing platform | |
| CN112882854B (en) | Method and device for processing request exception | |
| CN117555230B (en) | IO module multi-adapter control method and device and multi-channel IO module | |
| CN115292146B (en) | System capacity estimation method, system, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |