CN105138293A - Disk performance evaluating method based on database - Google Patents

Abstract

The invention discloses a disk performance evaluating method based on a database. According to the method, based on disk performance evaluation on the aspect of the database, the performance condition of a disk in a server on the practical application of the database is detected. The method specifically comprises the steps that a system is installed, and parameters of the system are configured, wherein an application server is prepared according to a practical application, the operation system is installed, and the parameters of the system are configured; the database is installed, and parameters of the database are configured, wherein a database GP is installed, and related user and environment variable parameters are configured; the database environment is debugged, wherein the database environment is configured according to system information, and related files are established to facilitate subsequent tests; the disk is configured, and the performance of the disk is tested; the disk needing to the tested is partitioned and mounted in the system, and a testing command is executed to carry out reading-writing operation on the disk; the reading-writing performance condition of the disc is checked according to a result; the implementing steps are completed in the server provided with the operation system with a linux kernel, and a user logs into the server with the root identity in operation.

Description

A kind of disk performance appraisal procedure based on database

Technical field

The present invention relates to computer server technical field, be specifically related to a kind of disk performance appraisal procedure based on database, a kind of flexibly, be applicable to the disk performance appraisal procedure of various server product.

Background technology

Along with the development of IT field technology, conventional informationization service and the increasingly powerful demand of cloud computing service to server become increasingly complex.As server product provider, the quality of a server, is whether can meet different application demands.Because the quantity of the disk on single machine gets more and more, the capacity of disk is increasing, and the impact of different disks on database application is different; In the case, for database application, just very necessary to the Performance Evaluation of disk.By contrast different disk, the data of different array, can find the disk being more applicable to database application, meet the application demand of client.

Hard disk performance index:

1. rotating speed

Rotating speed be in all indexs of hard disk except capacity the most noticeable performance parameter, with how many turns (RPM) per minute for unit, because rotating speed for hard disk transmission speed and continuous transmission speed most important, rotating speed is faster, and the speed that hard disk obtained and transmitted data is also faster.At present, the rotation speed of hard disk is mainly 7200RPM, 10000RPM and 15000RPM.

2. capacity

The storage of hard information on disk arranges in the form of concentric circles.Each circle is called a magnetic track.Magnetic track number in radial direction unit length is called track density Dt, and the information bit circumferentially in unit length is called bit density Db, and surface density Da is the product of track density and bit density, i.e. Da=DtxDb.Da shows that more greatly the quantity of information that a disc can store is larger.

The raising of surface density can make the data interference between adjacent track strengthen, and easily depart from when magnetic head carries out reading and writing data on magnetic track, mistake probability increases.The capacity of hard disk and disc number, surface density are in close relations, and the larger then capacity of these two numerical value is larger.But the increase of disc number can make hard disk volume thicken.The size of single dish capacity is directly connected to the size of DISK to Image capacity.Because 4 ~ 5 video discs generally can only be put in current hard disk, be only improved the capacity that the capacity often opening video disc just constantly can increase hard disk.But along with the raising of magnetic disc density, magnetic head just must be thereupon more and more sensitiveer.At present, single dish capacity all employs GMR magnetic head more than the hard disk of 20GB.

Single dish capacity except amount of capacity is had an impact, also and the data rate of hard disk have close relationship.The increase of track number is of great benefit to for the seek time reducing magnetic head, because the radius of magnetic sheet is fixing, the increase of track number mean track pitch from shortening, and magnetic head transfers to will shorten with regard to bit time needed for another magnetic track from a magnetic track, this will contribute to the raising of random data transmission speed.The growth of magnetic track internal linear magnetic density then has direct contact with the duration data transmission speed of hard disk, and GMR head technology of new generation ensure that this growth can not slow down because of the restriction of the sensitivity of magnetic head.So many times, the 7200RPM hard disk speed that the 5400RPM hard disk of higher single dish capacity can be lower than single dish capacity is more accelerated.

3. average seek time

Refer to the averaging time that magnetic head arrives target data place magnetic track average seek time, it directly affects the random data transmission speed of hard disk.Magnetic head average seek time, topmost determinative was the travelling speed of magnetic head power arm except outside the Pass having with single dish capacity.Current hard disk is approximately 7 ~ 9 milliseconds average seek time.

4. buffer memory

Buffer memory is also the considerable parameter of hard disk, and its size also directly can have influence on the overall performance of hard disk.In the reading process of data, the control chip in hard disk sends instruction, system directive is being read bunch the adjacent next one or the data of several bunches read in HCACHE.Like this, when system directive starts the data that will read next bunch time, hard disk does not just need to restart a read action, only needs the data in buffer memory to be sent in system hosts to have gone just.Therefore the increasing of buffer memory capacity can hold more pre-reads data.So greatly shorten the time of system wait.The buffer memory of current main flow hard disk reaches 2MB.

Because the influence factor of disk performance is too many, be difficult to, from the difference between the Data Comparison disk on surface, need to find rational means to contrast the data of first closing;

GREENPLUM is a relevant database cluster, and it is actually the logical data base be combined into by several independently database service.Comprise the instrument that an instrument may be used for test database connectedness in this database, we can pass through this instrument, carry out the network performance evaluation between cluster; Also can be used for assessing the network stabilization of each node between cluster.

Summary of the invention

The technical problem to be solved in the present invention is: based on above-mentioned technology, can by the test of database, carrys out the disk performance situation of disk of comprehensive assessment different capabilities, different rotating speeds.

The technical solution adopted in the present invention is:

Based on a disk performance appraisal procedure for database, described method is assessed based on the disk performance of database aspect, and in examination service device, disk is to the performance condition in database practical application, and detailed process is as follows:

One, installation system, configuration-system parameter: the application according to reality prepares application server, installing operating system, and configuration-system parameter;

Two, installation database configuration database parameter: installation database GP, and the relevant user of configuration and environmental variance parameter;

Three, tune-up data lab environment: according to system information configuration data lab environment, and set up relevant file, so that follow-up test;

Four, configure disk and carry out disk performance test: will the disk of test being needed to carry out subregion, and be mounted in system, perform test command and read-write operation is carried out to disk; The readwrite performance situation of disk is checked according to result;

Described performing step completes in the server of operating system being provided with linux kernel, and during operation, user carries out with root identity logs.

Described installation system, configuration-system parameter detailed process is as follows:

Installing operating system according to demand;

Kernel parameter is arranged:

Arrange in following parameters to/etc/sysctl.conf file, then run sysctl-p and make it come into force:

kernel.shmmax=500000000

kernel.shmmni=4096

kernel.shmall=4000000000

kernel.sem=2505120001002048

kernel.sysrq=1

kernel.core_uses_pid=1

kernel.msgmnb=65536

kernel.msgmax=65536

kernel.msgmni=2048

net.ipv4.tcp_syncookies=1

net.ipv4.ip_forward=0

net.ipv4.conf.default.accept_source_route=0

net.ipv4.tcp_tw_recycle=1

net.ipv4.tcp_max_syn_backlog=4096

net.ipv4.conf.all.arp_filter=1

net.ipv4.conf.default.arp_filter=1

net.ipv4.ip_local_port_range=102565535

net.core.netdev_max_backlog=10000

vm.overcommit_memory=2

Following parameters is set to/etc/security/limits.conf file:

*softnofile65536

*hardnofile65536

*softnproc131072

*hardnproc131072。

Described installation database configuration database parametric procedure is as follows:

Installation database;

Log on main frame with root and Greenplumbinaries is installed;

Create gpadmin user;

Create seg_hosts and all_hosts file to go forward side by side edlin;

Gpssh-exkeys is used to exchange key to root user;

Gpssh is used synchronously to arrange;

After SSH has configured, synchronously arrange with gpssh;

The user of amendment GreenPlum installation directory and owner;

Described tune-up data lab environment process is as follows:

The environmental variance of configuration gpadmin user;

Data catalogue is created to master and segment;

Gpssh-exkeys is used to exchange key to gpadmin user;

Synchro system clock.

Described configuration disk to perform disk performance test process as follows:

Be switched to the verification performing below root and run OS aspect;

Checking Disk bandwidth performance;

Output rusults.

Described methods and results analytic process is as follows:

By test result, judge that whether data are normal, if normally, carry out long pressure test.

Described method long-time stability evaluation process is as follows:

When single test is no problem, fill order repeatedly, for long disk pressure test, check its corresponding stability, see long operation, enough whether disk performance stablized.

Beneficial effect of the present invention is:

The present invention in different server products, different configurations, can support several operation systems: redhat, centos, suse run application; The method clear data, has good ease for use; Clear in structure between each module in program, is convenient to safeguard; And can this method be passed through, find whether server product has problems, and adjust accordingly.

The inventive method can be applied in various configuration, does not rely on hardware configuration, can carry out the disk performance assessment of different disk, different stage flexibly, easily, to meet the different needs.In addition, although this method configuration in early stage is complicated, test command is simple, and deploy content is relatively fixing, for being unfamiliar with linux and needing the technician carrying out operating under linux platform also can carry out the operation of being correlated with.

Accompanying drawing explanation

Fig. 1 is the inventive method realization flow figure.

Embodiment

Below according to Figure of description, in conjunction with embodiment, the present invention is further described:

Embodiment 1:

As shown in Figure 1, a kind of disk performance appraisal procedure based on database, described method is assessed based on the disk performance of database aspect, and in examination service device, disk is to the performance condition in database practical application, and detailed process is as follows:

One, installation system, configuration-system parameter: the application according to reality prepares application server, installing operating system, and configuration-system parameter;

Two, installation database configuration database parameter: installation database GP, and the relevant user of configuration and environmental variance parameter;

Three, tune-up data lab environment: according to system information configuration data lab environment, and set up relevant file, so that follow-up test;

Four, configure disk and carry out disk performance test: will the disk of test being needed to carry out subregion, and be mounted in system, perform test command and read-write operation is carried out to disk; The readwrite performance situation of disk is checked according to result;

Described performing step completes in the server of operating system being provided with linux kernel, and during operation, user carries out with root identity logs.

After test platform has been built, check that whether machine system information is correct, and make a record, so that for Data Comparison.

Embodiment 2:

On the basis of embodiment 1, installation system described in the present embodiment, configuration-system parameter detailed process is as follows:

Installing operating system according to demand, can support RHEL, Centos, SUSE sequence of maneuvers system; System recommendations are installed completely, are convenient to subsequent configuration systematic parameter;

Kernel parameter is arranged:

Arrange in following parameters to/etc/sysctl.conf file, then run sysctl-p and make it come into force; These parameters are its numeral of amendment not at will, after can strict verification when execution gpcheck, to find and wherein different, can report an error.

kernel.shmmax=500000000

kernel.shmmni=4096

kernel.shmall=4000000000

kernel.sem=2505120001002048

kernel.sysrq=1

kernel.core_uses_pid=1

kernel.msgmnb=65536

kernel.msgmax=65536

kernel.msgmni=2048

net.ipv4.tcp_syncookies=1

net.ipv4.ip_forward=0

net.ipv4.conf.default.accept_source_route=0

net.ipv4.tcp_tw_recycle=1

net.ipv4.tcp_max_syn_backlog=4096

net.ipv4.conf.all.arp_filter=1

net.ipv4.conf.default.arp_filter=1

net.ipv4.ip_local_port_range=102565535

net.core.netdev_max_backlog=10000

vm.overcommit_memory=2

Following parameters is set to/etc/security/limits.conf file;

*softnofile65536

*hardnofile65536

*softnproc131072

*hardnproc131072。

Embodiment 3:

On the basis of embodiment 1, installation database described in the present embodiment configuration database parametric procedure is as follows:

Installation database:

Log on main frame with root and Greenplumbinaries is installed;

Version name under #/bin/bashgreenplum-db-*.bin different system is different;

Create gpadmin user;

[rootgreenplumtmp]#useraddgpadmin

[rootgreenplumtmp]#passwdgpadmin

Changingpasswordforusergpadmin.

NewUNIXpassword:

BADPASSWORD:itisbasedonadictionaryword

RetypenewUNIXpassword:

passwd:allauthenticationtokensupdatedsuccessfully.

Create seg_hosts and all_hosts file to go forward side by side edlin;

Because the present embodiment only employs a station server, both do Master node, done again Segment node, therefore, as long as the content of these two files fills in a host name, below the environment of multiple stage, needed to modify according to actual environment;

Gpssh-exkeys is used to exchange key to root user;

[rootgreenplumgreenplum-db]#pwd

/usr/local/greenplum-db

[rootgreenplumgreenplum-db]#sourcegreenplum_path.sh

[rootgreenplumgreenplum-db]#cat/home/gpadmin/all_hosts

greenplum.localdomain

[rootgreenplumgreenplum-db]#gpssh-exkeys-f/home/gpadmin/all_hosts

[STEP1of5]createlocalIDandauthorizeonlocalhost

[STEP2of5]keyscanallhostsandupdateknown_hostsfile

[STEP3of5]authorizecurrentuseronremotehosts

[STEP4of5]determinecommonauthenticationfilecontent

[STEP5of5]copyauthenticationfilestoallremotehosts

[INFO]completedsuccessfully

Gpssh is used synchronously to arrange;

After SSH has configured, synchronously arrange with gpssh, such as, create user, synchronous installation medium etc.;

[rootgreenplumgreenplum-db]#gpssh-f/home/gpadmin/seg_hosts'useraddgpadmin'

[greenplum.localdomain]useradd:usergpadminexists

The user of amendment GreenPlum installation directory and owner;

[rootgreenplumlocal]#chown-Rgpadmin:gpadmingreenplum-db-4.1.1.8/

[rootgreenplumlocal]#chown-Rgpadmin:gpadmingreenplum-db

[rootgreenplumlocal]#ll

total84

drwxr-xr-x2rootroot4096May112011bin

drwxr-xr-x2rootroot4096May112011etc

drwxr-xr-x2rootroot4096May112011games

lrwxrwxrwx1gpadmingpadmin22Feb1921:30greenplum-db->./greenplum-db-4.1.1.8

drwxr-xr-x11gpadmingpadmin4096Feb1921:30greenplum-db-4.1.1.8

drwxr-xr-x2rootroot4096May112011include

drwxr-xr-x2rootroot4096May112011lib

drwxr-xr-x2rootroot4096May112011lib64

drwxr-xr-x2rootroot4096May112011libexec

drwxr-xr-x2rootroot4096May112011sbin

drwxr-xr-x4rootroot4096Feb1821:03share

drwxr-xr-x2rootroot4096May112011src

Embodiment 4:

On the basis of embodiment 1, described in the present embodiment, tune-up data lab environment process is as follows:

The environmental variance of configuration gpadmin user;

[gpadmingreenplum~]$cat.bashrc

#.bashrc

#Sourceglobaldefinitions

if[-f/etc/bashrc];then

./etc/bashrc

fi

#Userspecificaliasesandfunctions

source/usr/local/greenplum-db/greenplum_path.sh

Data catalogue is created to master and segment;

[rootgreenplumlocal]#mkdir/gpmaster

[rootgreenplumlocal]#chowngpadmin:gpadmin/gpmaster/

[rootgreenplumlocal]#mkdir/gpsegment1

[rootgreenplumlocal]#chowngpadmin:gpadmin/gpsegment1/

[rootgreenplumlocal]#mkdir/gpsegment2

[rootgreenplumlocal]#chowngpadmin:gpadmin/gpsegment2/

Gpssh-exkeys is used to exchange key to gpadmin user;

[gpadmingreenplum~]$id

uid=500(gpadmin)gid=500(gpadmin)groups=500(gpadmin)

[gpadmingreenplum~]$gpssh-exkeys-f/home/gpadmin/all_hosts

[STEP1of5]createlocalIDandauthorizeonlocalhost

[STEP2of5]keyscanallhostsandupdateknown_hostsfile

[STEP3of5]authorizecurrentuseronremotehosts

[STEP4of5]determinecommonauthenticationfilecontent

[STEP5of5]copyauthenticationfilestoallremotehosts

[INFO]completedsuccessfully

Synchro system clock;

[gpadmingreenplum~]$gpssh-fseg_hosts-vdate

[Reset...]

[INFO]logingreenplum.localdomain

[greenplum.localdomain]SunFeb1922:12:45CST2012

[INFO]completedsuccessfully

[Cleanup...]

Embodiment 5:

On the basis of embodiment 1, configure disk described in the present embodiment and to perform disk performance test process as follows:

Be switched to the verification performing below root and run OS aspect;

[rootgreenplumlocal]#gpcheck-f/home/gpadmin/all_hosts-mgreenplum.localdomain-sgreenplum.localdoamin

Checking Disk bandwidth performance;

[rootgreenplumlocal]#gpcheckperf-f/heome/gpadmin/seg_hosts-rds-D-d/gpsegment1-d/gpsegment2

Output rusults is as follows:

--------------------

--DISKWRITETEST

--------------------

--------------------

--DISKREADTEST

--------------------

==RESULT

====================

diskwriteavgtime(sec):107.08

diskwritetotbytes:33694318592

diskwritetotbandwidth(MB/s):300.09

diskwriteminbandwidth(MB/s):300.09[sdw]

diskwritemaxbandwidth(MB/s):300.09[sdw]

--perhostbandwidth--

diskwritebandwidth(MB/s):300.09[sdw]

diskreadavgtime(sec):72.46

diskreadtotbytes:33694318592

diskreadtotbandwidth(MB/s):443.46

diskreadminbandwidth(MB/s):443.46[sdw]

diskreadmaxbandwidth(MB/s):443.46[sdw]

--perhostbandwidth--

diskreadbandwidth(MB/s):443.46[sdw]

Embodiment 6:

On the basis of embodiment 1-5, described in the present embodiment, methods and results analytic process is as follows:

By test result, judge that whether data are normal, if normally, carry out long pressure test.

Embodiment 7:

On the basis of embodiment 6, described in the present embodiment, method long-time stability evaluation process is as follows:

When single test is no problem, fill order repeatedly, for long disk pressure test, check its corresponding stability, see long operation, enough whether disk performance stablized.

Above embodiment is only for illustration of the present invention; and be not limitation of the present invention; the those of ordinary skill of relevant technical field; without departing from the spirit and scope of the present invention; can also make a variety of changes and modification; therefore all equivalent technical schemes also belong to category of the present invention, and scope of patent protection of the present invention should be defined by the claims.

Claims (7)

1. based on a disk performance appraisal procedure for database, it is characterized in that: described method is assessed based on the disk performance of database aspect, and in examination service device, disk is to the performance condition in database practical application, and detailed process is as follows:

One, installation system, configuration-system parameter: the application according to reality prepares application server, installing operating system, and configuration-system parameter;

Two, installation database configuration database parameter: installation database GP, and the relevant user of configuration and environmental variance parameter;

Three, tune-up data lab environment: according to system information configuration data lab environment, and set up relevant file, so that follow-up test;

Four, configure disk and carry out disk performance test: will the disk of test being needed to carry out subregion, and be mounted in system, perform test command and read-write operation is carried out to disk; The readwrite performance situation of disk is checked according to result;

Described performing step completes in the server of operating system being provided with linux kernel, and during operation, user carries out with root identity logs.

2. a kind of disk performance appraisal procedure based on database according to claim 1, it is characterized in that: described installation system, configuration-system parameter detailed process is as follows:

Installing operating system according to demand;

Kernel parameter is arranged:

Arrange in following parameters to/etc/sysctl.conf file, then run sysctl-p and make it come into force:

kernel.shmmax=500000000

kernel.shmmni=4096

kernel.shmall=4000000000

kernel.sem=2505120001002048

kernel.sysrq=1

kernel.core_uses_pid=1

kernel.msgmnb=65536

kernel.msgmax=65536

kernel.msgmni=2048

net.ipv4.tcp_syncookies=1

net.ipv4.ip_forward=0

net.ipv4.conf.default.accept_source_route=0

net.ipv4.tcp_tw_recycle=1

net.ipv4.tcp_max_syn_backlog=4096

net.ipv4.conf.all.arp_filter=1

net.ipv4.conf.default.arp_filter=1

net.ipv4.ip_local_port_range=102565535

net.core.netdev_max_backlog=10000

vm.overcommit_memory=2

Following parameters is set to/etc/security/limits.conf file:

*softnofile65536

*hardnofile65536

*softnproc131072

*hardnproc131072。

3. a kind of disk performance appraisal procedure based on database according to claim 1, is characterized in that: described installation database configuration database parametric procedure is as follows:

Installation database;

Log on main frame with root and Greenplumbinaries is installed;

Create gpadmin user;

Create seg_hosts and all_hosts file to go forward side by side edlin;

Gpssh-exkeys is used to exchange key to root user;

Gpssh is used synchronously to arrange;

After SSH has configured, synchronously arrange with gpssh;

The user of amendment GreenPlum installation directory and owner.

4. a kind of disk performance appraisal procedure based on database according to claim 1, is characterized in that: described tune-up data lab environment process is as follows:

The environmental variance of configuration gpadmin user;

Data catalogue is created to master and segment;

Gpssh-exkeys is used to exchange key to gpadmin user;

Synchro system clock.

5. a kind of disk performance appraisal procedure based on database according to claim 1, is characterized in that: described configuration disk to perform disk performance test process as follows:

Be switched to the verification performing below root and run OS aspect;

Checking Disk bandwidth performance;

Output rusults.

6., according to the arbitrary described a kind of disk performance appraisal procedure based on database of claim 1-5, it is characterized in that, described methods and results analytic process is as follows:

By test result, judge that whether data are normal, if normally, carry out long pressure test.

7. a kind of disk performance appraisal procedure based on database according to claim 6, is characterized in that, described method long-time stability evaluation process is as follows:

When single test is no problem, fill order repeatedly, for long disk pressure test, check its corresponding stability, see long operation, enough whether disk performance stablized.