CN117687850A - Hard disk performance test method and device, electronic equipment and storage medium - Google Patents
Hard disk performance test method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN117687850A CN117687850A CN202311434021.XA CN202311434021A CN117687850A CN 117687850 A CN117687850 A CN 117687850A CN 202311434021 A CN202311434021 A CN 202311434021A CN 117687850 A CN117687850 A CN 117687850A
- Authority
- CN
- China
- Prior art keywords
- hard disk
- parameter combination
- test
- performance
- spdk
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000011056 performance test Methods 0.000 title claims abstract description 61
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000012360 testing method Methods 0.000 claims abstract description 162
- 238000012545 processing Methods 0.000 claims abstract description 56
- 238000011161 development Methods 0.000 claims abstract description 14
- 238000010606 normalization Methods 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 13
- 238000010998 test method Methods 0.000 claims 1
- 238000004422 calculation algorithm Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000011068 loading method Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Abstract
The invention provides a method, a device, electronic equipment and a storage medium for testing hard disk performance, which belong to the technical field of computers, wherein the method comprises the steps of switching a target server to a storage performance development Suite (SPDK) mode so as to run a hard disk drive code of the target server in a user mode and drive input/output (I/O) interrupt processing by adopting a polling mode; binding each hard disk of the target server with a corresponding CPU core of the central processing unit; in the SPDK mode, performing input/output I/O test on each hard disk within a preset time period, and determining an optimal parameter combination suitable for performing performance test on each hard disk; and carrying out limit performance test on each hard disk based on the optimal parameter combination. The invention can obtain more accurate limit performance index data.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method and apparatus for testing performance of a hard disk, an electronic device, and a storage medium.
Background
At present, with the increase of the calculation amount of the server, the storage server and the large data server are widely applied, the hard disk is widely applied in the actual environment, and the performance of the hard disk needs to be tested. At present, the hard disk is tested mainly in a kernel state, a hard disk performance test program is executed in the kernel state, a large amount of kernel context switching and interruption exist, a large amount of delay and expenditure are caused, and the limit performance data of the hard disk cannot be accurately reflected.
Disclosure of Invention
The invention provides a method and a device for testing hard disk performance, electronic equipment and a storage medium, which are used for solving the defects in the prior art that the hard disk is tested in a kernel state.
The invention provides a hard disk performance test method, which comprises the following steps:
switching the target server to a storage performance development Suite (SPDK) mode to run a hard disk drive code of the target server in a user state and drive input/output (I/O) interrupt processing by adopting a polling mode;
binding each hard disk of the target server with a corresponding CPU core of the central processing unit;
in the SPDK mode, performing I/O test on each hard disk within a preset time period, and determining an optimal parameter combination suitable for performing performance test on each hard disk;
and carrying out limit performance test on each hard disk based on the optimal parameter combination.
According to the method for testing the performance of the hard disk, in the SPDK mode, the I/O test is performed on each hard disk for a preset period of time, and the determination of the optimal parameter combination suitable for performing the performance test on each hard disk includes:
determining a plurality of parameter combinations;
according to each parameter combination, in the SPDK mode, I/O testing is conducted on each hard disk within a preset time period, bandwidth corresponding to each parameter combination and IOPS values of times of reading and writing operations per second are obtained, and the bandwidth corresponding to each parameter combination and the IOPS values are added after normalization processing respectively, so that performance data corresponding to each parameter combination are obtained;
And determining the maximum value in the performance data corresponding to all the parameter combinations, and taking the parameter combination corresponding to the maximum value as the optimal parameter combination.
According to the method for testing the performance of the hard disk, according to each parameter combination, in the SPDK mode, I/O testing is carried out on each hard disk within a preset time period, bandwidth and IOPS values corresponding to each parameter combination are obtained, the bandwidth and the IOPS values corresponding to each parameter combination are respectively normalized and added, and performance data corresponding to each parameter combination are obtained, wherein the method comprises the following steps:
according to each parameter combination, under the SPDK mode, sequentially reading and writing the first data block size in the preset time period to each hard disk to obtain the bandwidth of the sequentially reading test of the first data block size and the bandwidth of the sequentially writing test of the first data block size corresponding to each parameter combination;
according to each parameter combination, in the SPDK mode, performing random read and write test of the second data block size in a preset time period on each hard disk to obtain an IOPS value of the random read test of the second data block size and an IOPS value of the random write test of the second data block size corresponding to each parameter combination;
Normalizing the bandwidth of the sequential read test of the first data block size and the bandwidth of the sequential write test of the first data block size corresponding to each parameter combination to obtain the bandwidth of the sequential read write test of each parameter combination;
normalizing the IOPS value of the random read test of the second data block size and the IOPS value of the random write test of the second data block size corresponding to each parameter combination to obtain the IOPS value of the random read write test of each parameter combination;
and adding the bandwidth of the sequential read-write test of each parameter combination and the IOPS value of the random read-write test to obtain the corresponding performance data of each parameter combination.
According to the hard disk performance test method provided by the invention, the optimal parameter combination comprises the optimal queue depth and the optimal thread number.
According to the method for testing the performance of the hard disk, which is provided by the invention, the limit performance test is carried out on each hard disk based on the optimal parameter combination, and the method comprises the following steps:
and carrying out limit performance test on each hard disk according to the optimal queue depth and the optimal thread number.
According to the method for testing the performance of the hard disk, each hard disk of the target server is bound with a corresponding CPU core of the central processing unit, and the method comprises the following steps:
Determining the number of CPU cores corresponding to each hard disk according to the number of CPU cores of the target server and the number of hard disks of the target server;
and distributing the CPU cores of the target server according to the number of the CPU cores corresponding to each hard disk so as to bind each hard disk of the target server with the corresponding CPU core.
According to the method for testing the performance of the hard disk, the method for switching the target server to the SPDK mode comprises the following steps:
installing an SPDK;
after the SPDK is installed and compiled, performing test parameter setting on a fio test tool carried by the SPDK;
wherein the test parameter settings include: the exportLD_PRELOAD is used to introduce a fio path, ioengine is set to be SPDK, and the device address information which can be identified by the SPDK is specified through a parameter '-filename'.
The invention also provides a hard disk performance testing device, which comprises:
the initialization unit is used for switching the target server to a storage performance development Suite (SPDK) mode so as to run a hard disk drive code of the target server in a user state and drive input/output (I/O) interrupt processing by adopting a polling mode;
the binding core unit is used for binding each hard disk of the target server with the corresponding CPU core of the central processing unit;
The computing unit is used for carrying out I/O test on each hard disk within a preset time period under the SPDK mode, and determining an optimal parameter combination suitable for carrying out performance test on each hard disk;
and the testing unit is used for carrying out limit performance testing on each hard disk based on the optimal parameter combination.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the hard disk performance test method according to any one of the above when executing the program.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a hard disk performance test method as described in any of the above.
The invention also provides a computer program product comprising a computer program which when executed by a processor implements a hard disk performance testing method as described in any one of the above.
According to the hard disk performance testing method, the device, the electronic equipment and the storage medium, the target server is switched to the SPDK mode of the storage performance development suite, so that the hard disk drive code of the target server is operated in a user mode, the polling mode is adopted to drive the input/output I/O interrupt processing, delay and expenditure caused by context switching and interrupt can be avoided, a large amount of processing resources are saved, each hard disk of the target server is bound with the CPU core of the corresponding CPU, I/O requests on each hard disk can be timely processed, I/O processing efficiency is improved, the optimal parameter combination suitable for performance testing on each hard disk is determined through the I/O testing within the preset time length on each hard disk under the SPDK mode, and further the limit performance testing on each hard disk is carried out on the basis of the optimal parameter combination, so that more accurate limit performance index data can be obtained.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a method for testing performance of a hard disk according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a hard disk performance testing apparatus according to an embodiment of the present invention;
fig. 3 is a schematic entity structure diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Fig. 1 is a flow chart of a method for testing performance of a hard disk according to an embodiment of the present invention. As shown in fig. 1, the hard disk performance test method includes the following steps:
step 100, switching the target server to a storage performance development Suite (SPDK) mode to run the hard disk drive code of the target server in a user mode and adopting a polling mode to drive input/output (I/O) interrupt processing;
with the increasing computing power of the server, the demands of the storage server and the big data server for hard disks are increasing, and as a result, the NVME (Non-Volatile Memory Host Express, non-volatile memory host controller interface specification) hard disk is widely applied in the practical environment. With the rapid improvement of the performance of the NVME hard disk, some software cannot adapt to the faster and faster Input/Output (IO) read-write speed, which becomes a bottleneck of the system IO. When the central processing unit (Central Processing Unit, CPU) cannot respond to the interrupt signal in time when executing a large number of computing tasks or is busy, the read-write task may be severely affected. For example, the operating system kernel (Operating System kernel, OS kernel) requests a set of data, then loads the data into the cache of the hard disk, and sends an interrupt signal to the CPU, but the CPU core is busy processing other things, or is temporarily in a sleep state, or is reassigned to other tasks, and if the interrupt signal does not reach the CPU, the CPU will not get the IO data temporarily, so that the high-speed reading and writing of nvme cannot be fully performed. It can be understood from the above description that if the hard disk is tested in the kernel mode, there are a lot of kernel context switches and terminals, which cause a lot of delay and I/O overhead, so that the limit performance data of the hard disk cannot be reflected.
Wherein kernel mode refers to having access to all memory spaces and objects and the processor is not allowed to be preempted.
In order to solve the problem of testing the hard disk in the kernel state, the invention provides performance test of the hard disk in the SPDK mode.
SPDK (Storage Performance Development Kit) is a set of storage development kits designed specifically for specialized devices (e.g., NVMe SSD, etc.), which provide a range of high-performance, scalable, user-state underlying tools and libraries, and are currently being used in great numbers in the cloud computing field. SPDK has three major advantages: 1. the driving code of the storage device runs in the user mode instead of the kernel mode, so that the context switching between the kernel and the user mode is avoided, and the CPU overhead is saved; 2. the polling mode is driven, so that the IO (Input/Output) interrupt processing mode is changed, the IO request can be responded immediately after being submitted, interrupt wakeup can not be waited, and delay is reduced; 3. the use of locks is avoided in the IO path, while a message mechanism is employed, thereby avoiding the overhead of locks.
Therefore, in order to realize the performance test of the hard disk in the SPDK mode, the invention firstly switches the target server to the SPDK mode so as to run the hard disk drive code of the target server in a user state and adopts a polling mode to drive the input/output I/O interrupt processing.
The user mode refers to the mode that the user can run in the user process and cannot access the kernel data structure and the program of the operating system.
On the one hand, the hard disk drive code of the target server runs in the user mode, so that context switching and interruption can be avoided, which saves a great deal of processing overhead and allows more clock cycles to be used for actual data storage. No matter how complex the storage algorithm (e.g., de-redundancy, encryption, compression, blank block storage) is, wasting fewer clock cycles means better performance and latency.
On the other hand, the use of polling mode changes the basic model of conventional I/O. In the conventional I/O model, an application enters a sleep state after submitting a read-write request, and once the I/O is completed, an interrupt wakes up. The polling working modes are different, and after the application program submits the read-write request, other works are continuously executed, and whether the I/O is completed or not is checked back at a certain time interval. This approach avoids the delay and overhead of interrupts and allows the application to increase I/O efficiency.
Step 101, binding each hard disk of the target server with a corresponding CPU core of a central processing unit;
in the SPDK mode, the CPU does not execute an interrupt, but performs IO processing by using polling, so when performance test is performed on the hard disk, a large number of IOs reach the CPU at the same time, if no kernel binding is performed, the number of processing tasks of each CPU core is different due to the existence of a polling gap, at this time, a part of CPU cores may perform a large number of I/O processing, and a part of CPU cores are in the polling gap, thereby resulting in resource waste. Therefore, each hard disk of the target server needs to be bound with the corresponding CPU core of the central processing unit and then tested.
By binding each hard disk of the target server with the corresponding CPU core of the central processing unit, the I/O request on each hard disk can be processed in time.
102, in the SPDK mode, performing an I/O test on each hard disk within a preset period of time, and determining an optimal parameter combination suitable for performing a performance test on each hard disk;
specifically, important parameters that affect the I/O performance of the hard disk, such as queue depth, thread count, etc., need to be considered during testing. The different values of these important parameters constitute a plurality of parameter combinations.
In the SPDK mode, the invention carries out I/O test on each hard disk of the target server within a preset time period according to a plurality of parameter combinations, acquires performance data corresponding to the parameter combinations in the test process, and determines an optimal parameter combination suitable for carrying out performance test on each hard disk from the plurality of parameter combinations according to the acquired performance data.
Specifically, the parameter combination when the performance data takes the maximum value is taken as the optimal parameter combination.
It should be noted that the preset duration is usually a shorter duration, for example, 1 minute, and the present invention determines the optimal parameter combination through the I/O test in a short time.
The I/O tests include sequential read and write tests of a first data block size and random read and write tests of a second data block size. The first data block size is different from the second data block size.
And 103, carrying out limit performance test on each hard disk based on the optimal parameter combination.
It can be understood that after determining the optimal parameter combination suitable for performing performance test on each hard disk, performing limit performance test on each hard disk by adopting the optimal parameter combination can enable the hard disk to obtain the best performance, so that accurate limit performance data of each hard disk can be obtained. The ultimate performance test is to test the hard disk to obtain the ultimate performance index data of the hard disk.
According to the hard disk performance test method provided by the invention, the target server is switched to the SPDK mode of the storage performance development suite, so that the hard disk drive code of the target server is operated in a user mode, the polling mode is adopted to drive the input/output I/O interrupt processing, the delay and the expenditure caused by the context switching and the interrupt can be avoided, a large amount of processing resources are saved, each hard disk of the target server is bound with the CPU core of the corresponding CPU, the I/O request on each hard disk can be timely processed, the I/O processing efficiency is improved, the optimal parameter combination suitable for the performance test on each hard disk is determined by carrying out the I/O test within the preset time length on each hard disk in the SPDK mode, and further, the limit performance test is carried out on each hard disk based on the optimal parameter combination, so that more accurate limit performance index data can be obtained.
In some embodiments, the switching the target server to SPDK mode comprises:
installing an SPDK;
after the SPDK is installed and compiled, performing test parameter setting on a fio test tool carried by the SPDK;
wherein the test parameter settings include: the exportLD_PRELOAD is used to introduce a fio path, ioengine is set to be SPDK, and the device address information which can be identified by the SPDK is specified through a parameter '-filename'.
Specifically, before each hard disk of the target server is actually tested, the SPDK needs to be installed in the target server first, and the latest version can be downloaded in the network, and the set-with-fio can be used during installation, and the fio testing tool of the set-with-fio can be directly used.
After the compilation is installed, all its functions, such as viewing nvme devices, can be used. The kernel mode is switched to the user mode, in the mode, the CPU does not perform kernel context switching and interrupt, and then compared with the common fio, the test needs to set the following parameters: the exportld_reload is used to introduce a fio path, ioengine is SPDK, and the device address information which can be identified by the SPDK is specified by an additional parameter, filename'. The device address information here may be address information of each hard disk of the target server. The others are consistent with the normal fio. With the above arrangement, performance and pressure tests of the SPDK mode can be performed using the fio.
In the embodiment of the invention, the target server is switched to the SPDK mode so as to run the hard disk drive code of the target server in a user mode and drive the input/output I/O interrupt processing by adopting the polling mode, thereby avoiding delay and expenditure caused by context switching and interrupt, saving a large amount of processing resources and improving the accuracy of the hard disk limit performance test.
In some embodiments, the binding each hard disk of the target server with a corresponding central processing unit CPU core includes:
determining the number of CPU cores corresponding to each hard disk according to the number of CPU cores of the target server and the number of hard disks of the target server;
and distributing the CPU cores of the target server according to the number of the CPU cores corresponding to each hard disk so as to bind each hard disk of the target server with the corresponding CPU core.
Specifically, in order to bind each hard disk of the target server with the corresponding CPU core of the central processing unit, the number of CPU cores of the target server and the number of hard disks of the target server need to be obtained, and the number of CPU cores corresponding to each hard disk is determined according to the number of CPU cores of the target server and the number of hard disks of the target server.
Optionally, according to the number of CPU cores of the target server and the number of hard disks of the target server, the number of CPU cores corresponding to each hard disk is distributed evenly. For example, when a CPU of 32 cores performs a test of 4 disks, the number of cores allocated per disk is 8. Thus, each hard disk is guaranteed to have a CPU core processing its I/O tasks.
Then, the CPU cores of the target server are allocated according to the number of CPU cores corresponding to each hard disk, so that each hard disk of the target server is bound with the corresponding CPU core, for example, a CPU of 32 cores, CPU1 to CPU8 are allocated to hard disk 1, CPU9 to CPU16 are allocated to hard disk 2, CPU17 to CPU24 are allocated to hard disk 3, and CPU25 to CPU32 are allocated to hard disk 4.
In the embodiment of the invention, each hard disk of the target server is bound with the corresponding CPU core of the central processing unit, so that the I/O request on each hard disk can be processed in time, the I/O processing efficiency is improved, the resource waste is avoided, and the accuracy of the limit performance test of the hard disk can be improved.
In some embodiments, the optimal parameter combination includes an optimal queue depth and an optimal thread number.
Queue depth (iodepth) and thread (numjobs) numbers are important parameters that affect the hard disk I/O performance.
Queue depth setting is too small, queuing problems are caused, setting is too large and is not necessarily good, and if the storage capacity of the rear end is insufficient, the set queue depth is too large, and finally the overall IO performance is not good.
Given that each thread occupies very low CPU time for hard disk IO, within a certain number, multithreading may make the processing speed faster, but the efficiency of the promotion may not be very high. When the number of threads is increased to saturate the hard disk IO, the more threads are added, the more resources are allocated as the system is required, the slower the CPU response is, until the system has no redundant resources available for allocation and finally has no response. Therefore, the number of threads is also an important parameter affecting the I/O performance of the hard disk, and needs to be considered when performing the limit performance test on the hard disk.
In some embodiments, in the SPDK mode, performing an I/O test on each hard disk for a preset period of time, determining an optimal parameter combination suitable for performing a performance test on each hard disk includes:
determining a plurality of parameter combinations;
according to each parameter combination, in the SPDK mode, I/O testing is conducted on each hard disk within a preset time period, bandwidth corresponding to each parameter combination and IOPS values of times of reading and writing operations per second are obtained, and the bandwidth corresponding to each parameter combination and the IOPS values are added after normalization processing respectively, so that performance data corresponding to each parameter combination are obtained;
And determining the maximum value in the performance data corresponding to all the parameter combinations, and taking the parameter combination corresponding to the maximum value as the optimal parameter combination.
According to each parameter combination, in an SPDK mode, I/O testing is carried out on each hard disk within a preset time period, bandwidth corresponding to each parameter combination and the IOPS value of the times of reading and writing operations per second are obtained, and the bandwidth corresponding to each parameter combination and the IOPS value are respectively normalized and added to obtain performance data corresponding to each parameter combination; and then, determining the maximum value in the performance data corresponding to all the parameter combinations, and taking the parameter combination corresponding to the maximum value as the optimal parameter combination.
It can be understood that different parameter combinations and corresponding performance data are different, and the parameter combination corresponding to the maximum value in the performance data is used as an optimal parameter combination, so that the performance of the hard disk can be optimal, and then the ultimate performance of each hard disk is tested based on the optimal parameter combination, so that the ultimate performance index data of each hard disk can be obtained more accurately.
In general, the performance specification value of a hard disk is a 128K sequential read-write bandwidth value and a 4K random read-write IOPS value, where the IOPS value is far greater than the bandwidth value. In order to avoid the overlarge data gap, a normalization algorithm is needed to normalize the bandwidth and the IOPS value corresponding to each parameter combination, and then the bandwidth and the IOPS value can be added.
In some embodiments, each parameter combination includes a queue depth and a thread number.
Determining a plurality of parameter combinations consisting of queue depth and thread number;
and according to each parameter combination, in the SPDK mode, performing I/O test on each hard disk within a preset time period, and obtaining the bandwidth corresponding to each parameter combination and the IOPS value of the number of read-write operations per second, wherein the bandwidth corresponding to each parameter combination is the average value of a plurality of bandwidths corresponding to the parameter combination within the preset time period. The IOPS value for each parameter combination is an average of a plurality of IOPS values corresponding to the parameter combination within a preset time period.
The I/O test includes: sequential read and write testing of a first data block size, random read and write testing of a second data block size. The first data block size is different from the second data block size.
Further, respectively carrying out normalization processing on the bandwidth and the IOPS value corresponding to each parameter combination and then adding the bandwidth and the IOPS value to obtain performance data corresponding to each parameter combination; and determining the maximum value in the performance data corresponding to all the parameter combinations, and taking the parameter combination corresponding to the maximum value as the optimal parameter combination.
According to the hard disk performance test method provided by the invention, through carrying out the I/O test on each hard disk within the preset time period in the SPDK mode, the optimal parameter combination suitable for carrying out the performance test on each hard disk is determined, and further, based on the optimal parameter combination, the limit performance test is carried out on each hard disk, so that more accurate limit performance index data can be obtained.
In some embodiments, in the SPDK mode, performing an I/O test on each hard disk for a preset period of time according to each parameter combination, obtaining a bandwidth and an IOPS value corresponding to each parameter combination, and performing normalization processing on the bandwidth and the IOPS value corresponding to each parameter combination, and then adding to obtain performance data corresponding to each parameter combination, where the steps include:
according to each parameter combination, under the SPDK mode, sequentially reading and writing the first data block size in the preset time period to each hard disk to obtain the bandwidth of the sequentially reading test of the first data block size and the bandwidth of the sequentially writing test of the first data block size corresponding to each parameter combination;
according to each parameter combination, in the SPDK mode, performing random read and write test of the second data block size in a preset time period on each hard disk to obtain an IOPS value of the random read test of the second data block size and an IOPS value of the random write test of the second data block size corresponding to each parameter combination;
normalizing the bandwidth of the sequential read test of the first data block size and the bandwidth of the sequential write test of the first data block size corresponding to each parameter combination to obtain the bandwidth of the sequential read write test of each parameter combination;
Normalizing the IOPS value of the random read test of the second data block size and the IOPS value of the random write test of the second data block size corresponding to each parameter combination to obtain the IOPS value of the random read write test of each parameter combination;
and adding the bandwidth of the sequential read-write test of each parameter combination and the IOPS value of the random read-write test to obtain the corresponding performance data of each parameter combination.
Optionally, the normalization processing in the present invention may use a logarithmic function normalization algorithm, where a calculation formula of the logarithmic function normalization algorithm is as follows:
x'＝log10(x)/log10(max)；
where x is the original data and max is the maximum value in a set of data.
It should be noted that other normalization algorithms, such as polar normalization, zero-mean normalization, etc., may also be used. The invention does not limit the normalization algorithm.
For example, the hard disk performance is better when the product of the queue depth and the thread number is 256, and the combination of parameters of the queue depth and the thread number includes: (1, 256), (2, 128) (4, 64), …, (256, 1), and the same combination of parameters for the values of queue depth and thread count: (1, 1), (2, 2), (4, 4) … …. I.e., the number of threads and queue depth, includes: (1, 256), (2, 128) (4, 64), …, (256, 1), (1, 1), (2, 2), (4, 4) … ….
And according to each parameter combination, under the SPDK mode, sequentially reading and writing the first data block size in the preset time period to each hard disk to obtain the bandwidth of the sequentially reading test of the first data block size and the bandwidth of the sequentially writing test of the first data block size corresponding to each parameter combination. For example, the first preset duration is 1 minute, the first data block size is 128K, that is, in the SPDK mode, the sequential read and write test of 128K is performed on each hard disk for 1 minute, and the corresponding bandwidth in the test process is recorded. The average value of the bandwidths within the 1 minute is calculated as the bandwidth of the sequential read test of the first data block size and the bandwidth of the sequential write test of the first data block size corresponding to each parameter combination.
According to each parameter combination, in the SPDK mode, performing random read and write test of the second data block size in a preset time period on each hard disk to obtain an IOPS value of the random read test of the second data block size and an IOPS value of the random write test of the second data block size corresponding to each parameter combination; for example, the first preset duration is 1 minute, the second data block size is 4K, that is, in the SPDK mode, random read and write tests of 4K are performed on each hard disk for 1 minute, and the corresponding IOPS value in the test process is recorded. The average of the IOPS values over this 1 minute is calculated as the IOPS value of the random read test of the corresponding second data block size and the IOPS value of the random write test of the second data block size are combined for each parameter.
And normalizing the bandwidth of the sequential read test of the first data block size and the bandwidth of the sequential write test of the first data block size corresponding to each parameter combination and the IOPS value of the random read test of the second data block size and the IOPS value of the random write test of the first data block size corresponding to each parameter combination by adopting a normalization algorithm to obtain four values, and adding the four values to obtain the performance data corresponding to each parameter combination.
And comparing the performance data corresponding to all the parameter combinations, and taking the parameter combination when the performance data is maximum as the optimal parameter combination.
According to the hard disk performance test method provided by the invention, under the SPDK mode, the sequential read-write test of the first data block size and the random read-write test of the second data block size in the preset time period are carried out on each hard disk according to the multiple parameter combinations, the optimal parameter combination suitable for carrying out the performance test on each hard disk is determined, and further, the limit performance test is carried out on each hard disk based on the optimal parameter combination, so that more accurate limit performance index data can be obtained.
In some embodiments, the performing an ultimate performance test on each hard disk based on the optimal parameter combination includes:
And carrying out limit performance test on each hard disk according to the optimal queue depth and the optimal thread number.
It can be understood that after determining the optimal queue depth and the optimal thread number suitable for performing performance test on each hard disk, performing limit performance test on each hard disk by adopting the optimal queue depth and the optimal thread number can enable the hard disk to obtain the best performance, so that accurate limit performance data of each hard disk can be obtained.
The device for testing the performance of the hard disk provided by the invention is described below, and the device for testing the performance of the hard disk described below and the method for testing the performance of the hard disk described above can be correspondingly referred to each other.
Fig. 2 is a schematic structural diagram of a hard disk performance testing device according to an embodiment of the present invention. As shown in fig. 2, the hard disk performance test apparatus 200 includes:
an initialization unit 210, configured to switch the target server to a storage performance development suite SPDK mode, so as to operate a hard disk drive code of the target server in a user state and drive input/output I/O interrupt processing by adopting a polling mode;
a binding core unit 220, configured to bind each hard disk of the target server with a corresponding CPU core of the central processing unit;
A calculating unit 230, configured to perform an I/O test on each hard disk for a preset period of time in the SPDK mode, and determine an optimal parameter combination suitable for performing a performance test on each hard disk;
and the testing unit 240 is configured to perform an ultimate performance test on each hard disk based on the optimal parameter combination.
In some embodiments, in the SPDK mode, performing an I/O test on each hard disk for a preset period of time, determining an optimal parameter combination suitable for performing a performance test on each hard disk includes:
determining a plurality of parameter combinations;
according to each parameter combination, in the SPDK mode, I/O testing is conducted on each hard disk within a preset time period, bandwidth corresponding to each parameter combination and IOPS values of times of reading and writing operations per second are obtained, and the bandwidth corresponding to each parameter combination and the IOPS values are added after normalization processing respectively, so that performance data corresponding to each parameter combination are obtained;
and determining the maximum value in the performance data corresponding to all the parameter combinations, and taking the parameter combination corresponding to the maximum value as the optimal parameter combination.
In some embodiments, in the SPDK mode, performing an I/O test on each hard disk for a preset period of time according to each parameter combination, obtaining a bandwidth and an IOPS value corresponding to each parameter combination, and performing normalization processing on the bandwidth and the IOPS value corresponding to each parameter combination, and then adding to obtain performance data corresponding to each parameter combination, where the steps include:
According to each parameter combination, under the SPDK mode, sequentially reading and writing the first data block size in the preset time period to each hard disk to obtain the bandwidth of the sequentially reading test of the first data block size and the bandwidth of the sequentially writing test of the first data block size corresponding to each parameter combination;
according to each parameter combination, in the SPDK mode, performing random read and write test of the second data block size in a preset time period on each hard disk to obtain an IOPS value of the random read test of the second data block size and an IOPS value of the random write test of the second data block size corresponding to each parameter combination;
normalizing the bandwidth of the sequential read test of the first data block size and the bandwidth of the sequential write test of the first data block size corresponding to each parameter combination to obtain the bandwidth of the sequential read write test of each parameter combination;
normalizing the IOPS value of the random read test of the second data block size and the IOPS value of the random write test of the second data block size corresponding to each parameter combination to obtain the IOPS value of the random read write test of each parameter combination;
and adding the bandwidth of the sequential read-write test of each parameter combination and the IOPS value of the random read-write test to obtain the corresponding performance data of each parameter combination.
In some embodiments, the optimal parameter combination includes an optimal queue depth and an optimal thread number.
In some embodiments, the performing an ultimate performance test on each hard disk based on the optimal parameter combination includes:
and carrying out limit performance test on each hard disk according to the optimal queue depth and the optimal thread number.
In some embodiments, the binding each hard disk of the target server with a corresponding central processing unit CPU core includes:
determining the number of CPU cores corresponding to each hard disk according to the number of CPU cores of the target server and the number of hard disks of the target server;
and distributing the CPU cores of the target server according to the number of the CPU cores corresponding to each hard disk so as to bind each hard disk of the target server with the corresponding CPU core.
In some embodiments, the switching the target server to SPDK mode comprises:
installing an SPDK;
after the SPDK is installed and compiled, performing test parameter setting on a fio test tool carried by the SPDK;
wherein the test parameter settings include: the exportLD_PRELOAD is used to introduce a fio path, ioengine is set to be SPDK, and the device address information which can be identified by the SPDK is specified through a parameter '-filename'.
It should be noted that, the hard disk performance testing device provided by the embodiment of the present invention can implement all the method steps implemented by the embodiment of the method for testing hard disk performance, and can achieve the same technical effects, and the details of the same parts and beneficial effects as those of the embodiment of the method in the embodiment are not described in detail herein.
Fig. 3 illustrates a physical schematic diagram of an electronic device, as shown in fig. 3, where the electronic device may include: processor 310, communication interface (Communications Interface) 320, memory 330 and communication bus 340, wherein processor 310, communication interface 320, memory 330 accomplish communication with each other through communication bus 340. Processor 310 may invoke logic instructions in memory 330 to perform a hard disk performance test loading method comprising: switching the target server to a storage performance development Suite (SPDK) mode to run a hard disk drive code of the target server in a user state and drive input/output (I/O) interrupt processing by adopting a polling mode; binding each hard disk of the target server with a corresponding CPU core of the central processing unit; in the SPDK mode, performing I/O test on each hard disk within a preset time period, and determining an optimal parameter combination suitable for performing performance test on each hard disk; and carrying out limit performance test on each hard disk based on the optimal parameter combination.
Further, the logic instructions in the memory 330 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, the computer program product comprising a computer program, the computer program being storable on a non-transitory computer readable storage medium, the computer program, when executed by a processor, being capable of executing the hard disk performance test method provided by the above methods, the method comprising: switching the target server to a storage performance development Suite (SPDK) mode to run a hard disk drive code of the target server in a user state and drive input/output (I/O) interrupt processing by adopting a polling mode; binding each hard disk of the target server with a corresponding CPU core of the central processing unit; in the SPDK mode, performing I/O test on each hard disk within a preset time period, and determining an optimal parameter combination suitable for performing performance test on each hard disk; and carrying out limit performance test on each hard disk based on the optimal parameter combination.
In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform a hard disk performance test method provided by the above methods, the method comprising: switching the target server to a storage performance development Suite (SPDK) mode to run a hard disk drive code of the target server in a user state and drive input/output (I/O) interrupt processing by adopting a polling mode; binding each hard disk of the target server with a corresponding CPU core of the central processing unit; in the SPDK mode, performing I/O test on each hard disk within a preset time period, and determining an optimal parameter combination suitable for performing performance test on each hard disk; and carrying out limit performance test on each hard disk based on the optimal parameter combination.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. A method for testing performance of a hard disk, comprising:
switching the target server to a storage performance development Suite (SPDK) mode to run a hard disk drive code of the target server in a user state and drive input/output (I/O) interrupt processing by adopting a polling mode;
binding each hard disk of the target server with a corresponding CPU core of the central processing unit;
in the SPDK mode, performing I/O test on each hard disk within a preset time period, and determining an optimal parameter combination suitable for performing performance test on each hard disk;
and carrying out limit performance test on each hard disk based on the optimal parameter combination.
2. The method for testing performance of hard disks according to claim 1, wherein in the SPDK mode, performing I/O testing on each hard disk for a preset period of time, determining an optimal parameter combination suitable for performing performance testing on each hard disk, comprises:
determining a plurality of parameter combinations;
according to each parameter combination, in the SPDK mode, I/O testing is conducted on each hard disk within a preset time period, bandwidth corresponding to each parameter combination and IOPS values of times of reading and writing operations per second are obtained, and the bandwidth corresponding to each parameter combination and the IOPS values are added after normalization processing respectively, so that performance data corresponding to each parameter combination are obtained;
And determining the maximum value in the performance data corresponding to all the parameter combinations, and taking the parameter combination corresponding to the maximum value as the optimal parameter combination.
3. The method for testing the performance of a hard disk according to claim 2, wherein in the SPDK mode, performing an I/O test on each hard disk for a preset period of time according to each parameter combination, obtaining a bandwidth and an IOPS value corresponding to each parameter combination, respectively performing normalization processing on the bandwidth and the IOPS value corresponding to each parameter combination, and adding the normalized values to obtain performance data corresponding to each parameter combination, including:
according to each parameter combination, under the SPDK mode, sequentially reading and writing the first data block size in the preset time period to each hard disk to obtain the bandwidth of the sequentially reading test of the first data block size and the bandwidth of the sequentially writing test of the first data block size corresponding to each parameter combination;
according to each parameter combination, in the SPDK mode, performing random read and write test of the second data block size in a preset time period on each hard disk to obtain an IOPS value of the random read test of the second data block size and an IOPS value of the random write test of the second data block size corresponding to each parameter combination;
Normalizing the bandwidth of the sequential read test of the first data block size and the bandwidth of the sequential write test of the first data block size corresponding to each parameter combination to obtain the bandwidth of the sequential read write test of each parameter combination;
normalizing the IOPS value of the random read test of the second data block size and the IOPS value of the random write test of the second data block size corresponding to each parameter combination to obtain the IOPS value of the random read write test of each parameter combination;
and adding the bandwidth of the sequential read-write test of each parameter combination and the IOPS value of the random read-write test to obtain the corresponding performance data of each parameter combination.
4. A method of testing the performance of a hard disk according to any one of claims 1-3, wherein the optimal combination of parameters includes an optimal queue depth and an optimal thread count.
5. The method of claim 4, wherein said performing an ultimate performance test on each of said hard disks based on said optimal parameter combination comprises:
and carrying out limit performance test on each hard disk according to the optimal queue depth and the optimal thread number.
6. The method for testing the performance of the hard disk according to claim 1, wherein the binding each hard disk of the target server with the corresponding CPU core comprises:
Determining the number of CPU cores corresponding to each hard disk according to the number of CPU cores of the target server and the number of hard disks of the target server;
and distributing the CPU cores of the target server according to the number of the CPU cores corresponding to each hard disk so as to bind each hard disk of the target server with the corresponding CPU core.
7. The method for testing performance of a hard disk according to claim 1, wherein switching the target server to the SPDK mode comprises:
installing an SPDK;
after the SPDK is installed and compiled, performing test parameter setting on a fio test tool carried by the SPDK;
wherein the test parameter settings include: the exportLD_PRELOAD is used to introduce a fio path, ioengine is set to be SPDK, and the device address information which can be identified by the SPDK is specified through a parameter '-filename'.
8. A hard disk performance testing apparatus, comprising:
the initialization unit is used for switching the target server to a storage performance development Suite (SPDK) mode so as to run a hard disk drive code of the target server in a user state and drive input/output (I/O) interrupt processing by adopting a polling mode;
the binding core unit is used for binding each hard disk of the target server with the corresponding CPU core of the central processing unit;
The computing unit is used for carrying out I/O test on each hard disk within a preset time period under the SPDK mode, and determining an optimal parameter combination suitable for carrying out performance test on each hard disk;
and the testing unit is used for carrying out limit performance testing on each hard disk based on the optimal parameter combination.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the hard disk performance testing method of any one of claims 1 to 7 when the program is executed by the processor.
10. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the hard disk performance testing method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311434021.XA CN117687850A (en) | 2023-10-31 | 2023-10-31 | Hard disk performance test method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311434021.XA CN117687850A (en) | 2023-10-31 | 2023-10-31 | Hard disk performance test method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117687850A true CN117687850A (en) | 2024-03-12 |
Family
ID=90137903
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311434021.XA Pending CN117687850A (en) | 2023-10-31 | 2023-10-31 | Hard disk performance test method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117687850A (en) |
-
2023
- 2023-10-31 CN CN202311434021.XA patent/CN117687850A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Hady et al. | Platform storage performance with 3D XPoint technology | |
| EP3796179A1 (en) | System, apparatus and method for processing remote direct memory access operations with a device-attached memory | |
| US20130007488A1 (en) | Power management of a storage device including multiple processing cores | |
| US9489141B2 (en) | Efficient scheduling of Input/Output requests to reduce latency and maximize throughput in a flash storage device | |
| US11163452B2 (en) | Workload based device access | |
| US10241722B1 (en) | Proactive scheduling of background operations for solid state drives | |
| US11474712B2 (en) | Method, apparatus, device and storage medium for managing access request | |
| US20150236978A1 (en) | Managing resources in a distributed system using dynamic clusters | |
| US10296356B2 (en) | Implementation of reset functions in an SoC virtualized device | |
| US9721104B2 (en) | CPU-based measured boot | |
| US11194517B2 (en) | Method and apparatus for storage device latency/bandwidth self monitoring | |
| US11126375B2 (en) | Arbiter circuit for commands from multiple physical functions in a memory sub-system | |
| US11307801B2 (en) | Method, apparatus, device and storage medium for processing access request | |
| US10216423B1 (en) | Streams across multiple controllers to improve solid state drive performance | |
| US11231987B1 (en) | Debugging of memory operations | |
| US10216634B2 (en) | Cache directory processing method for multi-core processor system, and directory controller | |
| US10771580B1 (en) | Using machine learning to improve input/output performance of an application | |
| US10402101B2 (en) | System and method for using persistent memory to accelerate write performance | |
| CN112214162A (en) | Storage device and control method | |
| CN117687850A (en) | Hard disk performance test method and device, electronic equipment and storage medium | |
| CN116561091A (en) | Log storage method, device, equipment and readable storage medium | |
| CN111913662B (en) | SLC writing performance improving method and device, computer equipment and storage medium | |
| CN114816254A (en) | Hard disk data access method, device, equipment and medium | |
| Bougioukou et al. | Prototyping and performance evaluation of a dynamically adaptable block device driver for PCIe-based SSDs | |
| US11552892B2 (en) | Dynamic control of latency tolerance reporting values |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |