Detailed Description
Hereinafter, only certain exemplary embodiments are briefly described. As will be recognized by those of skill in the pertinent art, the described embodiments may be modified in various different ways without departing from the spirit or scope of the present disclosure. Accordingly, the drawings and description are to be regarded as illustrative in nature and not as restrictive.
Fig. 1 is a flow chart of a test method provided in an embodiment of the present disclosure. As shown in fig. 1, an embodiment of the present disclosure provides a test method, including:
s11, performing pressure test on the target storage cluster based on pressure test parameters of the target storage cluster;
And S12, determining a pressure test result of the target storage cluster based on the data difference degree between the write-in data and the read-out data of the pressure test and the performance change condition of the target storage cluster in the pressure test.
Wherein the target storage cluster is one of a plurality of storage clusters. The storage cluster includes a plurality of storage servers. For example, storage clusters are divided by location, storage clusters located in Beijing, storage clusters located in the North of the lake, and the like. Different storage clusters may have different versions of the service deployed. For example, the version of the service deployed by storage cluster a is different from the version of the service deployed by storage cluster B. For the storage clusters of different service versions, the pressure test can be performed respectively, so that the storage clusters of the service versions can meet the performance required by the pressure test.
Illustratively, the stress test parameters may include and travel the number of passes, the content and data amount of the input data for each pass, the stress test time and the number of stress tests, and the like.
In some embodiments, a test platform may be provided, where a target storage cluster and an input pressure test parameter are selected, and then a test execution button is clicked, where the test platform may call a corresponding test script according to the pressure test parameter, and perform a pressure test on the target storage cluster using the test script. Different pressure test parameters correspond to different test scripts.
In some embodiments, a default test script may be preset, and when the test script is applied, the test script is modified according to the pressure test parameters to obtain a target test script. And then, performing pressure test on the target storage cluster by using the target test script.
In some embodiments, a plurality of different test scripts may be preset according to different pressure test parameters. And when the method is applied, according to the input pressure test parameters, the corresponding test scripts are called to carry out pressure test on the target storage cluster.
The degree of data difference means, for example, the degree of difference in content and the degree of difference in data amount of written data and read data. If the write data and the read data are identical in content and the data amount is identical, the degree of difference between the two is zero.
Illustratively, the storage performance of the storage clusters may change during the stress test. Such as changes in the remaining storage space or footprint of the memory, and changes in data traffic.
In this example, the stress test is performed on the target storage cluster based on the stress test parameters of the target storage cluster, and the stress test result of the target storage cluster can be accurately determined based on the degree of data difference between the write data and the read data of the stress test and the performance change condition of the target storage cluster in the stress test.
In one embodiment, the pressure test parameter includes and travels the number of passes. In the step S11, the performing the stress test on the target storage cluster based on the stress test parameter of the target storage cluster includes:
And performing a corresponding number of storage operations on the target storage cluster according to the number of parallel processes to perform stress test on the target storage cluster. Wherein each storage operation includes a write operation and a read operation. Thus, each storage operation has corresponding write data and read data.
In practical application, the test platform can provide a pressure test window, and a tester can input pressure test parameters in the pressure test window, such as the number of parallel processes, a target storage cluster identifier, the input data quantity of each process, the pressure test starting time and the like.
When the test platform feeds back the pressure test result, the tester can adjust the input pressure test parameters according to the test result.
In some embodiments, the adjustment may also be automatically performed, and the adjustment may be stopped after the set pressure test target is reached.
In one embodiment, in the step S12, determining the stress test result of the target storage cluster based on the degree of data difference between the write data and the read data of the stress test and the performance variation of the target storage cluster in the stress test may include:
Determining the system stability degree of the target storage cluster based on the data difference degree between the writing data and the reading data of each storage operation in the pressure test;
in the pressure test, determining the flow variation amplitude and the memory variation amplitude of a target storage cluster;
And determining a pressure test result of the target storage cluster based on the system stability degree, the flow variation amplitude and the memory variation amplitude of the target storage cluster.
In this example, the system stability of the target storage cluster can be accurately determined according to the data difference between the write data and the read data of each storage operation in the pressure test, and the flow variation amplitude and the memory variation amplitude of the target storage cluster are determined in the test process, so that the pressure test result of the target storage cluster can be comprehensively evaluated based on the system stability, the flow variation amplitude and the memory variation amplitude of the target storage cluster, thereby facilitating the subsequent adjustment of the target storage cluster and improving the performance of the target storage cluster.
In some embodiments, the above-described stress testing step may be performed on each of a plurality of storage clusters, and then the preferred storage cluster is determined based on the stress test results for each storage cluster.
In some embodiments, according to a first algorithm, calculating a degree of data difference between the write data and the read data of each storage operation to obtain a first value; and determining the system stability degree of the target storage cluster based on the first numerical value. The first algorithm may be a weighted sum algorithm or a standard deviation algorithm, etc.
The magnitude of the flow change may be a difference between a maximum flow and a minimum flow of the target storage cluster during the stress test.
The memory variation amplitude may be a difference between a maximum memory occupancy value and a minimum memory occupancy value of the target storage cluster during the stress test.
In some embodiments, according to a second algorithm, calculating a system stability degree, a flow variation amplitude and a memory variation amplitude of the target storage cluster to obtain a second numerical value; and determining a stress test result of the target storage cluster based on the second value. For example, the second algorithm may be a weighted sum algorithm. And if the second value is larger than the set threshold value, determining that the pressure test result of the target storage cluster is passing. And if the second value is smaller than the set threshold value, determining that the pressure test result of the target storage cluster is failed.
In some embodiments, for the above-mentioned process of determining the pressure test result, corresponding thresholds may be set for the system stability, the flow variation amplitude and the memory variation amplitude, and whether each parameter meets the corresponding threshold is determined by the thresholds, if both meet the corresponding threshold, the pressure test result of the target storage cluster is determined to be passed, and if one of the pressure test results does not meet the corresponding threshold, the pressure test result of the target storage cluster is determined to be failed.
Illustratively, the determining the pressure test result of the target storage cluster based on the system stability, the flow variation amplitude and the memory variation amplitude of the target storage cluster includes one of the following steps:
Under the condition that the system stability, the flow variation amplitude and the memory variation amplitude of the target storage cluster all meet the set threshold, determining that the pressure test result of the target storage cluster is passed;
And under the condition that any one of the system stability degree, the flow variation amplitude and the memory variation amplitude of the target storage cluster does not meet the set threshold value, determining that the pressure test result of the target storage cluster is failed.
In this example, whether the pressure test result of the target storage cluster passes or not is determined according to the system stability degree, the flow variation amplitude and the memory variation amplitude of the target storage cluster, so that the pressure test result can be accurately judged.
In some embodiments, the critical pressure test parameters may be determined using the pressure test and the pressure test results. Specifically, the following description is given by taking the critical and running number as an example:
The method comprises the steps of updating and running the number of processes according to a set increasing function under the condition that a pressure test result is passing, and returning to execute a pressure test step until the pressure test result is not passing, and determining the critical and running number of the target storage cluster based on the updated and running number of processes and the increasing function;
the pressure testing step comprises the following steps:
Performing storage operation of corresponding quantity on the target storage cluster according to the number of the parallel processes so as to perform pressure test on the target storage cluster; and
And determining the stress test result of the target storage cluster based on the data difference degree between the writing data and the reading data of the stress test and the performance change condition of the target storage cluster in the stress test.
In this example, the increment value of the increment function may be 1, 2, 3, or the like, and the initial value may be the number of parallel processes of the last pressure test. The parallel process number of the last pressure test when the updating is stopped is used as the critical parallel process number of the target storage cluster, or the difference value between the parallel process number of the current pressure test when the updating is stopped and the increment value of the increment function is used as the critical parallel process number of the target storage cluster.
The critical and running number may be used to hint at an upper limit of the target storage cluster and running number.
The method comprises the steps of updating and running the number of processes according to a set decreasing function under the condition that a pressure test result is failed, and returning to execute a pressure test step until the pressure test result is passed, and determining the critical and running number of the target storage cluster based on the updated parallel number of processes;
the pressure testing step comprises the following steps:
Performing storage operation of corresponding quantity on the target storage cluster according to the number of parallel processes so as to perform pressure test on the target storage cluster; and
And determining the pressure test result of the target storage cluster based on the data difference degree between the writing data and the reading data of the pressure test and the performance change condition of the target storage cluster in the pressure test.
In this example, the decrementing value of the decrementing function may be 1,2, 3, etc., and the initial value may be the number of parallel processes of the last pressure test. The parallel process number of the pressure test when the update is stopped is taken as the critical parallel process number of the target storage cluster.
FIG. 2 is a block diagram of a pressure test platform according to an embodiment of the present disclosure. As shown in fig. 2, the pressure test platform includes a login module, a host information module, a host operation module, and a result module. The login module is mainly used for logging in the platform and acquiring the authority of the current user, displaying corresponding host information, and using the mysql database to verify whether the user is a registered user. The host information module mainly has the functions of adding host information, modifying host information, deleting host, checking host cpu, memory, IP address and the like, and performs operation, addition, deletion and check operation by using Python and mysql databases. The host operation module mainly has functions of logging in a host, executing commands, testing pressure and the like, and uses the Python built-in modules paramiko and unittest to test the logging in host and the pressure.
FIG. 3 is a flow chart of a pressure testing method of an embodiment of the present disclosure. As shown in fig. 3, in use, first, logging in, adding a host, modifying host information, viewing host information, and the like are performed, and then logging in a host and executing commands and pressure tests are performed.
As shown in fig. 4 to 9. Fig. 4 shows an interface diagram of a login test platform. Fig. 5 shows a front view of the test platform. Fig. 6 shows an addition interface diagram of host information. FIG. 7 illustrates a view interface diagram of host information. FIG. 8 illustrates an interface diagram for a host input command. Fig. 9 shows a parameter input interface diagram for the pressure test.
Fig. 10 is a block diagram of a test apparatus according to an embodiment of the present disclosure. As shown in fig. 10, the test apparatus includes:
The stress test module 110 is configured to perform stress test on the target storage cluster based on stress test parameters of the target storage cluster;
The test result determining module 120 is configured to determine a stress test result of the target storage cluster based on a degree of data difference between the write data and the read data of the stress test and a performance variation condition of the target storage cluster in the stress test.
In one embodiment, the pressure test parameter includes and runs the number of processes, and the pressure test module is specifically configured to:
And carrying out corresponding number of storage operations on the target storage cluster according to the parallel process number so as to carry out pressure test on the target storage cluster.
In one embodiment, the test result determination module includes:
A first determining unit configured to determine a system stability degree of the target storage cluster based on a degree of data difference between write data and read data of each storage operation in the stress test;
The second determining unit is used for determining the flow variation amplitude and the memory variation amplitude of the target storage cluster in the pressure test;
and the third determining unit is used for determining a pressure test result of the target storage cluster based on the system stability degree, the flow change amplitude and the memory change amplitude of the target storage cluster.
In one embodiment, the third determining unit is specifically configured to one of the following:
under the condition that the system stability, the flow variation amplitude and the memory variation amplitude of the target storage cluster all meet the set threshold, determining that the pressure test result of the target storage cluster is passing;
and under the condition that any one of the system stability degree, the flow variation amplitude and the memory variation amplitude of the target storage cluster does not meet a set threshold value, determining that the pressure test result of the target storage cluster is failed.
The functions of each unit, module or sub-module in each apparatus of the embodiments of the present disclosure may be referred to the corresponding descriptions in the above method embodiments, which are not repeated herein.
According to embodiments of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.
Fig. 11 illustrates a schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 11, the electronic device 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 802 or a computer program loaded from a storage unit 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. An input output (I/O) interface 805 is also connected to the bus 804.
Various components in electronic device 800 are connected to I/O interface 805, including: an input unit 806 such as a keyboard, mouse, etc.; an output unit 807 such as various types of displays, speakers, and the like; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, or the like. The communication unit 809 allows the electronic device 800 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.
The computing unit 801 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 801 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as test methods. For example, in some embodiments, the test method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 800 via the ROM 802 and/or the communication unit 809. When a computer program is loaded into RAM 803 and executed by computing unit 801, one or more steps of the test method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the test method by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer or other programmable atmosphere lamp fixture such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be carried out. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server incorporating a blockchain.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel, sequentially, or in a different order, provided that the desired results of the disclosed aspects are achieved, and are not limited herein.
The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.