KR100281928B1 - A Super RAID System using PC Clustering Technique - Google Patents

Abstract

In the present invention, two PCs are integrated into one system by using clustering technology to create a super RAID system having a hard disk capacity of 144 GB. The super RAID system hardware is mainly divided into a main body portion and a disk array portion. The main body consists of two Pentium PCs. In addition, the disk array uses 16 SCSI disks with a capacity of 9 GB. One of the PCs has a console like other PCs, but the other is a PC clustering consisting of a console-free system without a keyboard and monitor. It is a super raid system using technology.

Description

A super RAID system using PC clustering technique

The present invention relates to building a stable large file server system using a general PC.

An object of the present invention is to provide a large file server system of several hundred gigabytes and to perform stable operation of the system.

To achieve this goal, Super Raid consists of 8-10 SCSI disks with 9GB or 18GB of capacity per 9GB disk, and basic files for each disk to control a large number of disks using a PC. To build a system.

1 is a structural diagram of the superlaid system of the present invention

2 is a structural diagram of a hard disk portion of the present invention superlaid system

Figure 3 is a side view of the present invention superlaid system mounted on the cabinet

4 is a configuration diagram of a mode for determining a partition of the hard disk of the present invention;

5 is a state diagram making the present invention raid system

6 is a state diagram of the partition table of the present invention RAID zero;

7 is a configuration diagram of a file system of the present invention raid system

8 is a block diagram of the present invention, a regular online backup scrap.

<Explanation of symbols for main parts of the drawings>

Referring to the configuration and operation of the present invention in detail based on the accompanying drawings as follows.

1 is an internal structural diagram of a superlaid system of the present invention;

2 is a structural diagram of a hard disk portion of the present invention superlaid system,

3 is a side view of a state in which the superlaid system of the present invention is mounted in a cabinet;

4 is a configuration diagram of a mode for determining a partition of the hard disk of the present invention;

5 is a state diagram making the present invention raid system,

6 is a state diagram of a partition table of the present invention RAID zero;

7 is a configuration diagram of a file system of the present invention raid system,

8 shows a schematic diagram of the present invention's regular online backup scrap.

A RAID file system with 72GB of disk capacity was built using two PCs and integrated into one system using clustering technology to create a superlaid system with a total of 144GB.

The super RAID system hardware is composed of a main body and a disk DJP, the main body is 2 PCs, the disk array uses 16 SCSI disks of 9GB capacity, one console, the other keyboard and monitor It consists of a console that is not attached.

The operation of the present invention is as follows.

The main body of the file server has ASUS P54SPA and ASUS P2B as the main board and the master memory as the main memory, Pentium 333MH _Z and 150MH _Z, 128MB of two 64MB for the master server and 32MB for the slave server. Four were configured for 128MB.

As shown in FIG. 1, a CPU is stacked in a fan at the bottom right, and a VGA (video graphics array) adapter, a LAN card, and two SCSI controllers are sequentially attached from the bottom left to the top. The upper cabinet is equipped with a power supply, IDE hard disk drive, and profile disk driver. IDE type hard disks are the easiest to mount the system operating system, and the LAN card is 3 _C 595 Vortex 100baseTX, which is for input and output of large data.

The ASUS motherboard consists of four PCI slots and three ISA slots. The PCI slot has two SCSI cards, a LAN card and a VGA adapter. The two SCSI cards connect four 9GB SCSI disk drivers each to control a total of 72GB. Keyboards, VGA cards, monitors, floppy disk drives, and cases are the basic devices found on a typical PC.

In particular, slave servers are not equipped with keyboards and monitors, and VGA cards are needed to check the status of the file server. The case not only holds the body but also contains the power supply of the system. In this case, the tower type is adopted instead of the desktop type to facilitate the disassembly and assembly of the system. In particular, the capacity of the power supply unit is focused on the system expansion. The power supply for this case is 340W, which powers five SCSI disks.

The hard disk used in the FS144G-2 is a SCSI-based 9GB, consisting of eight ST93173N and eight ST1917N. The disks of the master server and the slave server connected four SCSI disk drivers to one SCSI controller. The SCSI controller consists of two AHA2940UW and two AH2940U. The AHA2940UW supports 15 different SCSI disk drivers at the same time by supporting the wide method, but only 4 disks are connected in consideration of system safety and performance of the system for data input and output. If you set the controller hard to 0, then the SCSI hard drive must be connected in the order of 1, 2, ... from the nearest one, and the terminator jumper must be adjusted to indicate that it is the last on the last device. In this way, four disks were connected, and eight disks were mounted in the disk array case as shown in FIG. 2.

Creating a device file in building a basic file system looks like this:

Standard Linux distributions such as Slackware and RedHat have a limited number of SCSI disk device files provided by default, which limits the number of disks that can be controlled per system. For example, the SCSI disk device files provided by Slackware are from / dev / sda to / dev / sdh, and the device files provided by RedHat are only from / dev / sda to / dev / sdg. In other words, if you use Slackware, you can control a total of eight SCSI disks. If you use RedHat, you can control only seven SCSI disks. Controlling large amounts of SCSI disks requires measures to overcome software limitations. This problem can be solved by manually creating the relevant device files. I've written a shell script that makesd a SCSI disk device file as a command.

The usage of the script is as follows.

# makesd device, where makesd is the name of the program and device is the sky device file you want to create. The Linux kernel can control up to 16 SCSI disks, from sda to sdp. The program creates both a raw device file and a device file that controls the first partition.

For example, the command makesd sdh to control the 8th SCSI disk will create sdh1 in addition to sdh. Of course, if a device file has already been created, it will not be duplicated. This program should be run from the system administrator account (root).

The file system construction is as follows.

To bundle multiple disks with RAID, you must first build a file system on each disk. Building file systems uses the Linux standard commands fdisk and mk2fs, which describe how to build these basic file systems.

The first program used to build a file system is fdisk. This program is usually used interactively and provides a lot of functionality. Among the various functions, there are six menus that are frequently used to build the file system for this super RAID: add partition, delete partition, view partition contents, cancel operation, save partition information disk, and view help.

For example, if you are building a partition on the system's first SCSI disk, / dev / sdh, you must first log in with an administrator account.

/ sbin / fdisk / dev / sdh

Enter the command. The first step in partitioning is to retrieve the existing partition information, which you can issue with the p command. Most new disks are partitioned for DOS. Therefore, the second task is to delete the existing partition, which is done by entering the d command and specifying the partition number to be deleted that the program asks interactively. Usually a newly purchased disk contains several partitions, and the d command must be repeated that many times. After deleting all existing partitions, use the n command to select the add partition option. The program asks if the partition type to be added is a primary partition or an extended partition. Answer p to indicate that it is the primary partition. When asked for the partition number to add, answer 1, answer the cylinder number at the beginning of the partition as 1, and the cylinder number at the end of the partition as the last cylinder number on the disk.

4 shows the contents of the dialogue with the fdisk program from deleting all existing partitions to creating a new basic partition and viewing the created partition information. To save the newly created partition information to disk, use the w command. This super RAID system has only one partition on each disk to maximize available space.

RAID improves I / O performance by tying several file systems together to form a large file system and enabling parallel I / O. Therefore, a separate file system must be built on each disk before building a RAID. Building a Linux file system uses the mke2fs program. For example, to build a basic file system in / dev / sdh1

# / sbin / mke2fs -c / dev / sdh1

You can give a command. Here `-c 'is an option that tells the program mke2fs to check for bad blocks and build a file system with only good blocks.

Note that changing the partition information on the disk or creating a new file system destroys all data currently on the disk. Therefore, fdisk or mke2fs programs need to be thoroughly checked before use.

Looking at the super raid deployment as follows.

Adding RAID Drives to the Kernel To build a Linux RAID, you need to add RAID drives to the kernel. The following explains how to add RAID to the kernel.

# Make menuconfig in the Linux kernel source directory / usr / src / linux on your system

Run the command to display the initial dialog box, Main Menu. Select the first option from the menu, `` Code maturity level options '', enter the relevant submenu, set the `` Prompt for development and / or incomplete code / drivers '' option, and exit back to the Main Menu.

In addition, go to the `` Floppy, IDE, and other block devices '' menu and select `` Multiple Devices driver support '' and RAID-0 as the built-in mode under Additional Block Devices (Figure 5). As shown in the figure, Linux supports various levels of RAID such as Linear, RAID-1, RAID-4 / 5 in addition to RAID-0, but this super RAID system basically uses RAID-0 to maximize I / O performance and available space. Adopted.

FIG. 5 shows how to compile a kernel with the make zImage command after finishing the compilation option selection process for the raid, and if the compilation is successful, install the newly created kernel with the make zlilo command. The new kernel, of course, only works after the system is restarted. If you selected module drivers during kernel configuration, you must run make modules followed by make modules \ _install to compile and install the modules.

Compile and install the RAID control tool as follows.

Once you have a kernel with a RAID drive, you need to install the software tools that make it work. The current Linux Raid tool is md035. However, md035 only works with Linux kernel 2.0.34 and earlier kernels. Since kernel version 2.0.35, compilation errors occur due to some software changes, so it is necessary to modify it to be able to compile with a new kernel.

Analysis of the source code revealed that four files, crc.c, mdadd.c, mdcreate.c, and mdparse.c, need to be modified. I created a patch file and indicated it in the appendix. Here's how to install the RAID control tool: First download md035 and unpack it in a suitable place on your system, for example / usr / local / src. The command used at this time

tar xvzf md035.tar.gz This command creates a directory called md035 and creates all the source files in that directory. Navigate to the md035 directory created next

% patch -p0 <../md035to036.diff will automatically modify four files: crc.c, mdadd.c, mdcreate.c, and mdparse.c.

After modifying the source files, compile and install.

% make

% su

# make install

The first command above compiles the raid tools. Normally two executables are created, mdcreate and mdadd.

The second command is to gain system administrator privileges (you must enter the system administrator password correctly when asked for a password, of course).

The third command installs the generated executables in the system area (/ sbin), links two commands, mdrun and mdstop, to mdadd, installs the madd manual, and finally finishes the four RAID device files / dev / md0, / dev Create / md1, / dev / md2 and / dev / md3.

RAID configuration and file system construction are as follows.

Raid configuration is done using mdcreate, one of the raid tools.

The general usage of the program mdcrate is

mdcreate [-c size] personality md \ _dev dev0 dev1 ...

Where size represents the size of the RAID chunk as the n power of the page size (page size is 4096 bytes on i386 Linux). The chunk size can also be expressed in units of 1,024 bytes by appending 'k' to the number. In this case, the chunk size must be a multiplier of two.

personality can be either linear, raid0, or raid1.

md \ _dev is the RAID device file you want to configure (eg / dev / md0), and dev0 .... are the basic disk device files that make up the blade, such as / dev / sda1 .... For example, to create / dev / md0 with four 9GB SCSI disks / dev / sda1 through / dev / sdd1 in 32 chunks of RAID0, create mdcreate -c32k raid0 / dev / md0 / dev / sda1 / dev / sdb1 / give the command dev / sdc1 / dev / sdd1 The mdcreate command keeps the execution result in a file called / etc / mdtab. FIG. 6 shows the contents of the file of the system where the real blade is built.

6 illustrates a chunk. The chunk size is related to the input / output performance of the RAID device. The chunk size should be determined according to the average size of a file input / output to the file system and the number of physical devices configuring the RAID. In other words, multiply the chunk size by the number of disks to make it similar to the average file size. If the chunk size is too large, it is possible to use only some disks intensively instead of parallelism, whereas if the chunk size is too small, multiple I / Os will be performed on the same disk. On the other hand, when the system has a large number of processes running simultaneously, the load balance occurs automatically as disk I / O occurs almost randomly.

After creating an mdtab using the program mdcreate, the mdadd command registers the newly configured blade with the kernel and starts it up.

mdadd -ar

You can give a command like this:

Creating a file system on a built RAID device omits the fdisk operation, unlike building a basic file system. In addition, the damaged block check is already performed when the basic system is built, so there is no need to do it separately. So the process of building a file system on / dev / md0, creating a mount point / u0 and mounting it there

mke2fs -b 4096 / dev / md0

mkdir / u0

mount / dev / md0 / u0

Same as In the first command mke2fs above, the option -b 4096 tells you to create a file system with a block size of 4,096 bytes.

The standard Linux block size is 1,024 bytes. The reason for the large block size is to consider the performance and stability of large file systems.

For reference, the standard block size for the Cray C90 system is 4,096.

The fstab registration of the RAID file system is as follows.

To mount your RAID file system automatically or semi-automatically at system boot time, you need the system's fstab and its associated shell scripts.

Figure 7 shows the contents of / etc / fstab including two RAID file systems, md0 and md1. The first column of fstab is the device name, the second column is the mount point, the third column is the file system type, and the fourth column is the mount option. The fifth column is the backup switch used by the dump program, and the sixth column is the mount order of the file system.

FIG. 7 illustrates a file system. The characteristic of the fstab is that `noauto 'is added to the mount option so that the RAID file system is not automatically mounted at system boot. Of course, it is possible to mount a RAID file system automatically, but automatic mounts are disadvantageous for slave hosts operating in a clustering environment, which is discussed in the boot procedure in System Management.

The system management of the cluster environment is as follows.

In a cluster environment, the console only has a master server and the rest of the slave servers check the boot status over the network. It would be very inconvenient if the slave servers in the cluster system went down due to an unexpected power outage and then failed the file system check on subsequent boot and went into single user mode. In this case, you have to connect the console to the system that failed to boot and go through a relatively complex file system recovery process. Therefore, in a clustered environment, it is convenient to check only the minimum file system needed to bring the system up and, if it is right, switch to multi-user mode and bring the network up to restore other file systems from the network connection. In other words, Raid Super RAID file system is required to perform file system check and mount in a separate script after switching to multi-user mode without automounting at boot time.

We have written a shell script, rc.md, that registers the RAID device with the kernel at system startup, checks the file system, and performs a series of tasks to mount it if everything is OK. The general usage of the script is

rc.md [-m module \ _name]

If the kernel has a built-in RAID drive, say rc.md. If you compiled a RAID-0 drive as a module, give it rc.md -m raid0.o. The script registers the RAID device with the kernel, then checks / etc / mdtab and mounts the RAID file systems listed one after the other. The rc.md command is usually inserted at the very beginning of rc.local. The success of the system boot can be checked directly by the central system with the console by viewing the messages on the console, but other hosts without the console should be monitored by remote monitoring. As a remote monitoring method, it is possible to think first of how to check the remote login after the host, which is inconvenient because it involves the login procedure. Another method of remote monitoring is to collect system logs of all hosts in a cluster and centrally monitor them in one place. This method is convenient for monitoring the cluster system because only one log host needs to be logged in and the other hosts do not need to be logged in. For security reasons, how to send the system log to a separate log host and to receive the system log of another computer is omitted here for security reasons, see the syslogd and syslog.conf manuals.

The system closure is as follows.

In general, a system shutdown means one of two things: a system halt or a system reboot. If you perform a system stop, the computer stops all operations and waits for power to shut down. If you need to replace computer parts or shut down for a long time, it's time to stop the system. Restarting the system is the process of restarting the system after stopping all currently running processes. System restarts are performed during preventive maintenance or software changes.

On the other hand, there may be an emergency that requires all hosts in Gloucester to be stopped quickly. But in order to stop the system, each host has a remote login.

shutdown -h now

It can be very cumbersome and time consuming to stop the system using the iso command. Therefore, a method to close the remote computer immediately without a remote login procedure must be devised. We devised a way to use a specific TCP port as a way to quickly stop all computers in a clustered environment. This method uses TCP ports 67 and 68 of the host for remote closure. In other words, if a specific host specified requires a TCP connection to these ports, it will immediately execute the system shutdown commands `shutdown -h now 'or` shutdown -r now' without any login procedure.

To configure this environment, you need to modify four files in the / etc directory: services, inetd.conf, hosts.allow, hosts.deny.

That is, in the services file

reboot 67 / tcp # shutdown -r now

halt 68 / tcp # shutdown -h now

Add two lines, such as `reboot 'to alias TCP port 67 and` halt' to alias TCP port 68. Originally, these ports were reserved for BOOTP servers and BOOTP clients, and this super RAID system did not use them for that purpose and chose these port numbers because they were closely related to booting.

The second is in the inetd.conf file

reboot stream tcp nowait root / usr / sbin / tcpd / sbin / shutdown -r now

halt stream tcp nowait root / usr / sbin / tcpd / sbin / shutdown -h now

Add two lines. This is a configuration that starts a tcpwrapper named / usr / sbin / tcpd when a TCP connection request comes in on port 67 or 68 over the network.

The program tcpd examines the host-specific access rights defined in hosts.allow and hosts.deny, and if the host requesting access is granted, `/ sbin / shutdown -r now 'or` / sbin / shutdown -h now Will execute each command.

Of course, after modifying the inetd.conf file, restarting the inetd daemon will have more than the effect.

Note: Making such changes to inetd.conf is related to system security. You must work with the help of a professional.

On the other hand, shutting down a system is an act of shutting down the computer, so its authority needs to be restricted. This permission is granted and restricted by tcpwrapper through two files: hosts.allow and hosts.deny. For example, if you only give access to local hosts and hosts with IP addresses 1.1.1.10 and 1.1.1.11, and restrict access to all other hosts, you can use hosts.allow

ALL: 127.0.0.1

ALL: 1.1.1.10, 1.1.1.11

Make it look like, hosts.deny

ALL: ALL

You can make it like this: For more information about granting and restricting access by host or domain using tcpwrapper, see the system online manual, which is issued with the command `man hosts \ _access'.

When the environment is configured as above, the command to close the remote host is

telnet hostname reboot

telnet hostname halt

You can use one of two commands. The former is used to restart the system and the latter to stop the system.

File system recovery is as follows.

Unexpected power outages sometimes occur when using the system. If you suddenly lose power, the contents of the computer's memory will be lost instead of being written to disk. On the other hand, as Linux system performs disk data I / O in async mode, there is a significant time delay instead of instantaneous data I / O. In the end, sudden power failure can lead to corruption of the disk file system. Linux always checks whether a file system has been closed normally at system boot time and, if it is not found, performs recovery of that file system. The automatic recovery performed by the system will only succeed if the disk corruption is minor.

There are two types of file system corruptions: root file system corruption and other file system corruption. If the root file system is damaged and normal recovery failed, boot from the rescue disk and repair it. If that doesn't work, you have to reinstall the OS. To reduce damage to the root partition, place only the minimum number of files needed to boot and manage your system. That is, files that are frequently updated during system operation should be stored outside the root partition to minimize the possibility of damage to the root partition. Meanwhile, most file system corruption occurs on other partitions. If a damaged partition is found, the system will show the damage to the console and wait for a login from the console. To recover from a crash, log in to the administrator account (root) at the console and issue a command such as `fsck -y device} ', where device is the name of the partition to be recovered.Note: Partition recovery must be umounted. Should be done at

Therefore, the present invention, the superlaid system is useful not only for seismic data processing but also for data backup of general computer processing center. In addition, the system's capacity of 144 GB is comparable to the 200 GB disk capacity of the nation's largest supercomputer Cray-C90, and its assembly cost does not exceed the price of a regular PC. This system has been used by the Korea Resource Research Institute for the actual seismic data processing for the past two years. On the other hand, Kreonet's Internet file server was created in April 1997 with a very similar specification to the system. Until now (98.12), it has been providing stable service without any problem to domestic and international Internet users. In general computer center, this system can be used as online backup server.

Claims (3)

A RAID file system with 72GB of disk capacity was built using two PCs and integrated into one system using clustering technology to create a superlaid system with a total of 144GB. The super RAID system hardware is composed of a main body and a disk DJP, the main body is 2 PCs, the disk array uses 16 SCSI disks of 9GB capacity, one console, the other keyboard and monitor Super Raid system using PC clustering technology, characterized by consisting of a console without attachment. The main body of the file server selects the master and slave servers as the motherboards of the ASUS P54SPA and ASUS P2B, respectively, the main storage unit is Pentium 333MH _Z and 150MH _Z. Super RAID system using PC clustering technology, which consists of 128MB of four 32MB. At the bottom right of the cabinet, the CPU is stacked in the fan, and from the bottom left to the top, a video graphics array (VGA) adapter, a LAN card, and two SCSI controllers are attached in turn, and the upper cabinet is a power supply, an IDE hard disk. Drive, PROFI disk drive, IDE type hard disk, LAN card consists of 3 _C 595 Vortex 100baseTX, ASUS motherboard consists of 4 PCI slots and 3 ISA slots, and 2 SCSI cards and LAN card in PCI slot It is equipped with VGA adapter, and two SCSI cards connect four 9GB SCSI disk drives to control 72GB of total capacity, and super RAID system using PC clustering technology.