CN108776638A

CN108776638A - A kind of server runs the method and system of complete machine fio pressure tests

Info

Publication number: CN108776638A
Application number: CN201810594597.5A
Authority: CN
Inventors: 彭笑笑
Original assignee: Zhengzhou Yunhai Information Technology Co Ltd
Current assignee: Zhengzhou Yunhai Information Technology Co Ltd
Priority date: 2018-06-11
Filing date: 2018-06-11
Publication date: 2018-11-09

Abstract

The present invention provides a kind of method and system of server race complete machine fio pressure tests, includes the following steps：Create disk pressure test script；Testing time is set, disk is tested by formatting, sets the testing time that the test block under various patterns voluntarily adjusts each module, realizes the adjusting of complete machine fio testing times.The drive information of all disks of acquisition system；The drive information of acquisition is screened, classified, is sorted.Test script is called to carry out pressure test all disks of disc-pack.Monitor the operating condition of disk fio；Judgement has often executed the calling of a script, jumps out this cycle at once and continues next cycle.Detection and the log information for collecting test process；Information is completed in output test.

Description

A kind of server runs the method and system of complete machine fio pressure tests

Technical field

The invention belongs to server technology fields, and in particular to a kind of server run complete machine fio pressure tests method and System.

Background technology

As big data, cloud computing flourish, the market demand of massive store server is continuously increased, big at present Capacity storage server can configure a variety of storages, and the demand to some pressure tests of storage is also enriched constantly.

Existing method carries out fio tests to disk, some carry out single-deck and test fio, and the script of some polydiscs test is mostly Disk number and drive has been fixedly installed, has needed to configure disk in manual modification test script according to system disk before test Number and drive etc., cannot achieve full automation.More and complicated system is configured for disk, existing method will be unable to At fio complete machine pressure tests.

Invention content

In order to overcome above-mentioned deficiency in the prior art, the present invention provides a kind of server and runs complete machine fio pressure tests Method and system, to solve the above technical problems.

The technical scheme is that:

A kind of method that server runs complete machine fio pressure tests, includes the following steps：

Create disk pressure test script；

Test script is called to carry out pressure test all disks of disc-pack.

Further, step creates disk pressure test script, specifically includes：

The drive information of all disks of acquisition system；

The drive information of acquisition is screened, classified, is sorted.

Further, include before the drive information of step acquisition all disks of system：

Testing time is set, realizes that process is as follows：

Disk is tested by formatting, sets the testing time that the test block under various patterns voluntarily adjusts each module, is realized The adjusting of complete machine fio testing times.

Further, step calls test script to carry out pressure test all disks of disc-pack, further includes：

Monitor the operating condition of disk fio；

Judgement has often executed the calling of a script, jumps out this cycle at once and continues next cycle.

Further, further include after step calls test script progress pressure test to all disks of disc-pack：

Detection and the log information for collecting test process；

Information is completed in output test.

Technical solution of the present invention also provides the system that a kind of server runs complete machine fio pressure tests, including creation module and Execution module；

Creation module, for creating disk pressure test script；

Execution module calls test script to carry out pressure test for all disks to disc-pack.

Further, creation module includes acquiring unit and processing unit；

Acquiring unit, the drive information for obtaining all disks of system；

Processing unit, the drive information for being obtained to acquiring unit are screened, are classified, are sorted.

Further, creation module further includes setting unit and operating unit；

Operating unit, for formatting test disk；

Setting unit voluntarily adjusts the testing time of each module for setting the test block under various patterns, realizes complete machine fio The adjusting of testing time.

Further, execution module further includes monitoring unit and judging unit；

Monitoring unit, the operating condition for monitoring disk fio；

Judging unit, the calling for judging often to have executed a script jump out this cycle and continue next cycle at once, real Now all disks are carried out at the same time fio pressure tests.

Further, creation module further includes detection and collector unit and output unit；

Detection and collector unit, the log information for detecting and collecting test process；

Output unit completes information for exporting test.

Each drive in the test script searching loop disc-pack that the present invention creates calls pressure for each disk Script, and/dev/null catalogues are imported, forbid standard output, carries out running background so that often executed the tune of a script With jumping out this cycle at once and continue next cycle, realize that all disks are carried out at the same time fio pressure tests.It is set according to test It fixes time, auto-collection system test log after the completion of running background is tested.

As can be seen from the above technical solutions, the present invention has the following advantages：To all disks in system while fio pressure Test, while can select without testing disk（Such as system disk）, and the testing time can be freely adjusted, disk is matched It sets complicated fio system tests and manpower and resource is greatly saved.

In addition, design principle of the present invention is reliable, and it is simple in structure, there is very extensive application prospect.

It can be seen that compared with prior art, the present invention with substantive distinguishing features outstanding and significantly improving, implementation Advantageous effect be also obvious.

Description of the drawings

Fig. 1 is the method flow diagram that a kind of server provided in an embodiment of the present invention runs complete machine fio pressure tests.

Specific implementation mode

The present invention will be described in detail below in conjunction with the accompanying drawings and by specific embodiment, and following embodiment is to the present invention Explanation, and the invention is not limited in following implementation.

As shown in Figure 1, the method that a kind of server provided in an embodiment of the present invention runs complete machine fio pressure tests, including such as Lower step：

S1：Create disk pressure test script；

S11：Disk is tested by formatting, sets the testing time that the test block under various patterns voluntarily adjusts each module, Realize the adjusting of complete machine fio testing times；

S12：The drive information of all disks of acquisition system；

S13：The drive information of acquisition is screened, classified, is sorted.

S2：Test script is called to carry out pressure test all disks of disc-pack.

S21：Monitor the operating condition of disk fio；

S22：Judgement has often executed the calling of a script, jumps out this cycle at once and continues next cycle.

S3：Detection and the log information for collecting test process；

S4：Information is completed in output test.

The embodiment of the present invention also provides the system that a kind of server runs complete machine fio pressure tests, including creation module and holds Row module；

Creation module, for creating disk pressure test script；

Creation module includes acquiring unit, processing unit, setting unit and operating unit；

Acquiring unit, the drive information for obtaining all disks of system；

Operating unit, for formatting test disk；

Execution module further includes monitoring unit and judging unit；

Monitoring unit, the operating condition for monitoring disk fio；

Creation module further includes detection and collector unit and output unit；

Output unit completes information for exporting test.

Based on above-described embodiment：

Invention uses the drive of lsblk related command extraction system all types disks；

It is sorted to disk sort with sort related commands.

In addition it is not required to disk to be tested for certain（Such as system disk）, deleted from disc-pack using sed related commands It removes.It is carried out at the same time fio pressure tests finally for all disks in disc-pack.

The parameter that time parameter starts script start.sh as test is set, the testing time is voluntarily adjusted.

Each drive in invention searching loop disc-pack calls pressure script for each disk, and imports/dev/ Null catalogues forbid standard output, carry out running background so that the calling for often having executed a fio.sh jumps out this at once Cycle continues next cycle, realizes that all disks are carried out at the same time fio pressure tests.According to the testing setup time, transported on backstage Auto-collection system test log after the completion of row fio.

Operation ./stop.sh can stop fio tests at any time.

Detailed process is as follows：

Script test.sh

#!/bin/bash

time=$(date +%Y-%m-%d-%H-%M)

if [ -n "$2" ];then

TIME=$2

else

TIME=3600

fi

DIR=`pwd`

>/var/log/messages

>/var/log/dmesg

By TIME variables, realizes the voluntarily adjusting of testing time, be defaulted as 3600s.

if [ ! -e /lib64/libaio.so.1 ] ; then

cp $DIR/Tool/libaio.so.1.0.1 /lib64/

ln -s /lib64/libaio.so.1.0.1 /lib64/libaio.so.1

fi

rm -f Log/*

cd Log/

DiskA=$1

DiskB=$3

export TIME

export A

export B

disk=$(lsblk |sed '/NAME/d' |awk '{print $1}' |sort |sed '/'$DiskA'/d')

if [ -n "$DiskB" ];then

disk=$(echo "$disk" |sed '/'$DiskB'/d')

fi

All disk drives of acquisition system, classification and ordination delete the disk that need not carry out pressure test from disk list disk It removes.

for i in $disk

do

export i

export j

export DIR

$DIR/fio.sh > /dev/null &

done

Fio.sh is called to carry out pressure test all disks of disc-pack, and in running background.Because all fio tests are equal In running background, order iostat-t 1 can be used to be monitored the operating condition of disk fio during the test.

t=`expr $TIME+30`

t=$[t*58]

sleep $t

cat /var/log/messages|grep -i error >> messages_error.txt

cat /var/log/messages|grep -i failed >> messages_error.txt

cat /var/log/dmesg|grep -i error >> dmesg_error.txt

cat /var/log/dmesg|grep -i failed >> dmesg_error.txt

cd /var/log

cp messages dmesg messages_error.txt messages_error.txt dmesg_error.txt dmesg_error.txt $DIR/Log

echo -e "\033[32m All tests have completed!\033[0m"

It waits for pressure operation in backstage to complete, carries out the detection and collection of system log；

Export " All tests have completed!"

Script fio.sh

#!/bin/bash

echo -e "\033[32m $i start fio test\033[0m"

mkfs.ext4 -T largefile -F /dev/$i

#Sequential read and write test

echo "mode,bws,iops" >> fio_result_$i.csv

for j in {1k,2k,4k,8k,16k,32k,64k,128k,256k,512k,1024k,2048k}

do

echo "$i-$j-read testing"

$DIR/Tool/fio -filename=/dev/$i -direct=1 -iodepth 32 -rw=read -ioengine= libaio -bs=$j -numjobs=1 --time_based --runtime=$TIME --norandommap -- randrepeat=0 --bandwidth-log=1 -group_reporting -name=mytest >$i-$j-read.txt

BW=`cat $i-$j-read.txt |grep iops=|cut -d ',' -f 2|grep -oE "[0-9.]+"`

BWU=`cat $i-$j-read.txt |grep iops=|cut -d ',' -f 2|cut -d '=' -f 2|sed s'/[0-9.]//'g`

IOPS=`cat $i-$j-read.txt |grep iops=|cut -d ',' -f 3|cut -d '=' -f 2`

echo "$i-$j-read,$BW($BWU),$IOPS" >>fio_result_$i.csv

sleep 30

echo "$i-$j-write testing"

$DIR/Tool/fio -filename=/dev/$i -direct=1 -iodepth 32 -rw=write -ioengine =libaio -bs=$j -numjobs=1 --time_based --runtime=$TIME --norandommap -- randrepeat=0 --bandwidth-log=1 -group_reporting -name=mytest >$i-$j-write.txt

BW=`cat $i-$j-write.txt |grep iops=|cut -d ',' -f 2|grep -oE "[0-9.]+"`

BWU=`cat $i-$j-write.txt |grep iops=|cut -d ',' -f 2|cut -d '=' -f 2|sed s'/[0-9.]//'g`

IOPS=`cat $i-$j-write.txt |grep iops=|cut -d ',' -f 3|cut -d '=' -f 2`

echo "$i-$j-write,$BW($BWU),$IOPS" >>fio_result_$i.csv

#rm -rf $i-$j.txt

sleep 30

done

#Randomread and Randomwrite test

echo " " >> fio_result_$i.csv

for j in {1k,2k,4k,8k,16k,32k,64k,128k,256k,512k,1024k,2048k}

do

echo "$i-$j-randread testing"

$DIR/Tool/fio -filename=/dev/$i -direct=1 -iodepth 32 -rw=randread - ioengine=libaio -bs=$j -numjobs=1 --time_based --runtime=$TIME --norandommap --randrepeat=0 --bandwidth-log=1 -group_reporting -name=mytest >$i-$j- randread.txt

BW=`cat $i-$j-randread.txt |grep iops=|cut -d ',' -f 2|grep -oE "[0-9.]+" `

BWU=`cat $i-$j-randread.txt |grep iops=|cut -d ',' -f 2|cut -d '=' -f 2| sed s'/[0-9.]//'g`

IOPS=`cat $i-$j-randread.txt |grep iops=|cut -d ',' -f 3|cut -d '=' -f 2`

echo "$j-randread,$BW($BWU),$IOPS" >>fio_result_$i.csv

#rm -rf $i-$j-randread.txt

sleep 30

echo "$i-$j-randwrite testing"

$DIR/Tool/fio -filename=/dev/$i -direct=1 -iodepth 32 -rw=randwrite - ioengine=libaio -bs=$j -numjobs=1 --time_based --runtime=$TIME --norandommap --randrepeat=0 --bandwidth-log=1 -group_reporting -name=mytest >$i-$j- randwrite.txt

BW=`cat $i-$j-randwrite.txt |grep iops=|cut -d ',' -f 2|grep -oE "[0-9.] +"`

BWU=`cat $i-$j-randwrite.txt |grep iops=|cut -d ',' -f 2|cut -d '=' -f 2| sed s'/[0-9.]//'g`

IOPS=`cat $i-$j-randwrite.txt |grep iops=|cut -d ',' -f 3|cut -d '=' -f 2 `

echo "$i-$j-randwrite,$BW($BWU),$IOPS" >>fio_result_$i.csv

#rm -rf $i-$j-randwrite.txt

sleep 30

done

#Mixed Random read and write test

echo " " >> fio_result_$i.csv

for j in {4k,8k,16k,32k,64k,128k,256k,512k,1024k,2048k}

do

echo "$i-$j-randrw testing"

$DIR/Tool/fio -filename=/dev/$i -direct=1 -iodepth 32 -rw=randrw -- rwmixread=70 -ioengine=libaio -bs=$j -numjobs=1 --time_based --runtime=$TIME --norandommap --randrepeat=0 --bandwidth-log=1 -group_reporting -name=mytest >$i-$j-randrw.txt

BW=`cat $i-$j-randrw.txt |grep iops=|cut -d ',' -f 2|grep -oE "[0-9.]+"`

BWU=`cat $i-$j-randrw.txt |grep iops=|cut -d ',' -f 2|cut -d '=' -f 2|sed s'/[0-9.]//'g`

IOPS=`cat $i-$j-randrw.txt |grep iops=|cut -d ',' -f 3|cut -d '=' -f 2`

echo "$j-randrw,$BW($BWU),$IOPS" >>fio_result_$i.csv

#rm -rf $i-$j-randrw.txt

sleep 30

done

Test disk is formatted, the test block under various patterns is set, each module is voluntarily adjusted using global variable TIME Testing time, to realize the adjusting of complete machine fio testing times.The result of each test block is saved in corresponding log.

Script stop.sh

#!/bin/bash

killall –g fio

cp /var/log/message $DIR/Log

cp /var/log/dmesg $DIR/Log

Operation stop.sh stops all fio tests manually.

Term " first ", " second ", " third " " in description and claims of this specification and above-mentioned attached drawing Four " etc.（If there is）It is for distinguishing similar object, without being used to describe specific sequence or precedence.It should manage The data that solution uses in this way can be interchanged in the appropriate case, so as to the embodiment of the present invention described herein can in addition to Here the sequence other than those of diagram or description is implemented.In addition, term " comprising " and " having " and their any deformation, It is intended to cover and non-exclusive includes.

The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, as defined herein General Principle can be realized in other embodiments without departing from the spirit or scope of the present invention.Therefore, of the invention It is not intended to be limited to the embodiments shown herein, and is to fit to and the principles and novel features disclosed herein phase one The widest range caused.

Claims

1. a kind of method that server runs complete machine fio pressure tests, which is characterized in that include the following steps：

Create disk pressure test script；

Test script is called to carry out pressure test all disks of disc-pack.

2. the method that a kind of server according to claim 1 runs complete machine fio pressure tests, which is characterized in that step is created Disk pressure test script is built, is specifically included：

The drive information of all disks of acquisition system；

The drive information of acquisition is screened, classified, is sorted.

3. the method that a kind of server according to claim 2 runs complete machine fio pressure tests, which is characterized in that step obtains It takes before the drive information of all disks of system and includes：

Testing time is set, realizes that process is as follows：

4. the method that a kind of server according to claim 1 runs complete machine fio pressure tests, which is characterized in that step pair All disks of disc-pack call test script to carry out pressure test, further include：

Monitor the operating condition of disk fio；

5. the method that a kind of server according to claim 1 runs complete machine fio pressure tests, which is characterized in that step pair All disks of disc-pack call test script to carry out pressure test：

Detection and the log information for collecting test process；

Information is completed in output test.

6. the system that a kind of server runs complete machine fio pressure tests, which is characterized in that including creation module and execution module；

Creation module, for creating disk pressure test script；

7. the system that a kind of server according to claim 6 runs complete machine fio pressure tests, which is characterized in that create mould Block includes acquiring unit and processing unit；

Acquiring unit, the drive information for obtaining all disks of system；

8. the system that a kind of server according to claim 7 runs complete machine fio pressure tests, which is characterized in that create mould Block further includes setting unit and operating unit；

Operating unit, for formatting test disk；

9. the system that a kind of server according to claim 6 runs complete machine fio pressure tests, which is characterized in that execute mould Block further includes monitoring unit and judging unit；

Monitoring unit, the operating condition for monitoring disk fio；

10. the system that a kind of server according to claim 6 runs complete machine fio pressure tests, which is characterized in that create mould Block further includes detection and collector unit and output unit；

Output unit completes information for exporting test.