JP2009230673A - Computer, computer system, distributed processing method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To effectively utilize various hardware resources as grid computing environment even when a general application which is not specialized to the grid computing environment is used. <P>SOLUTION: For a computer which is connected to other computers to allow processing by grid computing, software for grid for attaining the processing by the grid computing is provided below a kernel in a hierarchy of software and above hardware. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a computer, a computer system, a distributed processing method, and a program that can perform processing by grid computing by connecting to another computer.

  2. Description of the Related Art In recent years, attention has been focused on grid computing that increases the computing capacity of the entire system by configuring a system by combining a plurality of relatively inexpensive PCs (Personal Computers) and servers as a grid.

  In recent years, CPUs (Central Processing Units) have become multicore, and symmetric multiprocessing (SMP) hardware has been provided at low cost. For this reason, in general commercial application software, many products that take the SMP environment into consideration are sold.

An example of a general grid computing environment is shown in FIG.
As shown in FIG. 11, in a general computer system that performs processing by grid computing, a software function for performing communication and control between grids is implemented in the form of middleware in a layer above the OS (Operating System). is doing.

Further, as a related technology of the present invention, an auxiliary system for building grid computing is included in the software part of the microprocessor, and the auxiliary system cooperates with grid computing construction support software existing in the middleware. (For example, refer to Patent Document 1).
JP 2005-56408 A

However, in the general grid computing environment described above, application software specialized for grid computing has to be created individually in order to demonstrate the computing power of hardware, and it was not highly versatile. .
In general applications, it has been difficult to use a plurality of grids across and continuously by grid computing for disk resources and memory resources that can be used by the applications.

In addition, since the grid computing environment and the SMP environment are not similar, application software that considers operation in a widely distributed SMP environment can exhibit its capabilities in the grid computing environment. There wasn't.
In addition, most of these general commercial application software is difficult to use not only for CPU resources built in a grid computing environment but also for large-capacity memory resources such as disk resources and memory resources.

  As described above, the operation on the grid computing environment is not taken into consideration, and the normal application considering the operation on the general SMP environment has a CPU on the network provided by the grid computing environment. The various resources such as memory could not be utilized.

  In addition, the above-mentioned Patent Document 1 tries to make an operation efficient by linking an auxiliary system such as a security auxiliary system for authenticating a microprocessor with the above-described grid computing construction support software in the middleware. In addition, it is not even considered to make it possible to effectively use each hardware resource in a grid computing environment by using an application for a general SMP environment that is not specialized in the grid computing environment. There wasn't.

  The present invention has been made in view of such circumstances, and even when a general application that is not specialized for a grid computing environment is used, various hardware resources are used as the grid computing environment. It is an object of the present invention to provide a computer, a computer system, a distributed processing method, and a program that can be used effectively.

  In order to achieve such an object, a computer according to the present invention is a computer that is connected to other computers and can perform processing by grid computing, and is below the kernel in the software hierarchy and above the hardware. Grid software for realizing the processing by the grid computing is provided.

  Further, the computer system according to the present invention is characterized in that a plurality of the computers according to the present invention described above are connected to each other and can be processed by grid computing.

  Further, the distributed processing method according to the present invention is a distributed processing method in a computer system capable of processing by grid computing by connecting a plurality of computers, and when performing processing by an application by the grid computing, It is characterized by using grid software provided below the kernel in the hierarchy and above the hardware.

  The program according to the present invention is a computer program that can be connected to other computers and can be processed by grid computing, and is for a grid provided below the kernel in the software hierarchy and above the hardware. The computer causes the computer to execute processing by the application in the grid computing.

  As described above, according to the present invention, various hardware resources can be effectively used as a grid computing environment even when a general application that is not specialized for the grid computing environment is used. Can do.

Next, an embodiment to which a computer, a computer system, a distributed processing method, and a program according to the present invention are applied will be described in detail with reference to the drawings.
The computer system of the present embodiment exemplifies a suitable system that exhibits the function of application software for the SMP environment that is widely distributed in the grid computing environment.

First, an outline of the present embodiment will be described.
In this embodiment, as shown in FIG. 1, a computer is connected to another computer to form a computer system, and processing by grid computing is possible. Grid software for realizing processing by grid computing is provided below the kernel and above the hardware in the computer software hierarchy.

  More specifically, in the present embodiment, even in a general application that is not specialized for a grid computing environment, various hardware resources can be effectively used as a grid computing execution environment. As shown in FIG. 2, software functions for performing communication and control between grids are implemented in a layer closer to the one-stage hardware (HW) than the OS kernel layer.

  With this configuration, in this embodiment, each computer resource linked in the grid computing environment can be provided to a higher-level application as if it were a single computer. Even a normal application considering operation in the SMP environment can enjoy the benefits of multiple CPUs and large-capacity memory resources realized in the grid environment.

Next, a schematic configuration of the present embodiment will be described with reference to FIGS.
In recent years, it has become mainstream to provide a hardware virtualization layer (Hardware Abstraction Layer; HAL) between the hardware and the kernel in the hierarchical structure of software for the purpose of absorbing differences due to hardware architecture. Yes.

  In the PC grid execution environment proposed as the present embodiment, the grid software is arranged in any of the forms shown in FIGS. About this arrangement | positioning, it is possible to select suitably by conditions, such as processing speed, the ease of mounting, and the ease of control.

  However, when considering using a general commercial operating system, it is normal for the operating system provider to provide the HAL. Therefore, it is not possible to newly define the processing between the HAL and the kernel as shown in FIG. Since it is not realistic and the realization of FIG. 5 in which the HAL and the grid software are integrated is also not realistic, the hierarchical structure of FIG. 4 is the easiest to implement. Therefore, in the following description, the configuration shown in FIG. 4 will be described as an example.

Next, the configuration of the computer as the present embodiment will be described with reference to FIG.
The computer hardware 120 includes CPU cores 123 and 124, a memory 126, an HBA (Host Bus Adapter) 125, a disk (Hard Disk) 127, a NIC (Network Interface Card) 121, and a high-speed NIC 122. . The disk 127 is connected to the hardware main body (main board) via the HBA 125.

  The NIC includes a high-speed NIC 122 as a high-speed NIC specialized for performing communication between grids, in addition to the NIC 121 that performs communication for exhibiting a function as a normal computer. As for the high-speed NIC 122, for example, 10 Gigabit Ethernet (registered trademark) or Infini Band is suitable for implementation at present.

As software for realizing the functions of this embodiment, in addition to the OS 102, kernel 103, and HAL 104 provided by the operating system provider, grid software 110 is provided in a layer below the HAL 104.
The grid software 110 includes an inter-grid CPU arbitration function 111, an inter-grid memory arbitration function 112, and an inter-grid disk arbitration function 113 as a characteristic configuration of the present embodiment.

  In the computer according to the present embodiment, the application 101 runs on the OS 102 in the environment described above. In the present embodiment, the application 101 has a function of utilizing a plurality of CPU cores 123 and 124 in parallel by multithreading or multitasking.

Next, the operation of the computer system as the present embodiment will be described with reference to FIG.
In the configuration example of the computer system shown in FIG. 7, for the sake of easy understanding, an operation in an environment in which two computers of PC 301 and PC 302 are connected will be described as an example.

The inter-grid CPU arbitration functions 111 and 211 of the PC 301 and the PC 302 communicate with each other through both high-speed NICs 122 and 222, and make the kernel recognize the existence of both CPU cores via the HAL. That is, the CPU recognizes all CPU cores included in the computer system according to the present embodiment.
In the example of FIG. 7, the kernels 103 and 203 can recognize the presence of a total of four CPU cores of the PC 301 and the PC 302.

The inter-grid memory arbitration functions 112 and 212 of the PC 301 and the PC 302 communicate with each other through both high-speed NICs 122 and 222, and cause the kernel to recognize the presence of both memories via the HAL. That is, the kernel recognizes all the memories included in the computer system according to the present embodiment.
In the example of FIG. 7, the existence of the total memory amount of the PC 301 and the PC 302 can be recognized by the kernels 103 and 203.

The inter-grid disk arbitration functions 113 and 213 of the PC 301 and the PC 302 communicate with each other through both high-speed NICs 122 and 222, and make the kernel recognize the presence of both disks via the HAL. That is, the kernel recognizes all the disks included in the computer system according to this embodiment.
Disks can be recognized as physical disk drives as they are, or multiple physical disk drives can be operated in tandem to configure RAID (Redundant Array of Inexpensive Disks) to ensure availability and large files It is also possible to create a system.

  Here, the case where the normal application 101 running on the PC 301 requests the OS to provide a service will be described with reference to the flowchart of FIG.

  When the normal application 101 requests memory allocation, the OS 102 receives a memory reservation request instruction set and issues a memory reservation request to the kernel 103 (step S1). The kernel 103 requests the HAL 104 to secure the memory, and the HAL 104 requests the memory securing function 112 in the grid software 110 to secure the memory.

  The inter-grid memory arbitration function 112 manages the allocation status of physical memory as a hardware resource of the PC 301, and when there is a sufficient margin in the usage status of the physical memory 126 of the PC 301 (step S2; Yes), the memory space of the PC 301 Is secured (step S3), and information regarding the memory space actually secured is notified to the HAL 104, which is the software of the upper layer.

  When the inter-grid memory arbitration function 112 determines that a necessary and sufficient margin cannot be secured in the physical memory 126 of the PC 301 (step S2; No), the inter-grid memory arbitration function 212 of the PC 302 and the inter-grid memory arbitration function 212 Communication is performed and a request for securing the memory is made to the PC 302 (step S4).

  Upon receiving the request, the inter-grid memory arbitration function 212 of the PC 302 manages the securing status of the physical memory 226 as the hardware resource of the PC 302, and secures the memory space of the PC 302 when there is a surplus in the usage status of the physical memory of the PC 302. (Step S5), information about the actually reserved memory space is notified to the HAL 104 of the PC 301 via the inter-grid memory arbitration function 112 of the PC 301 (step S6).

  With the above-described series of operation flows, the physical memory space of the PC 301 and the PC 302 is recognized by the normal application of the PC 301 as if they were an integrated memory space, and a vast physical memory space that cannot be realized by a single hardware. Can be used.

  Next, a case where the normal application 101 running on the PC 301 requests CPU resources from the OS will be described with reference to the flowchart of FIG.

  The kernel 103 defines the number of CPU resources that can be used via the HAL at startup (step S11). When the kernel 103 is started on the PC 301, the inter-grid CPU arbitration function 111 communicates with the inter-grid CPU arbitration function 211 of the PC 302, recognizes a total of four CPU cores of both the PC 301 and the PC 302 as available resources, and starts the OS 102. (Step S12).

  When a new thread or process is defined while the normal application 101 is operating (step S13; Yes), a new CPU resource definition request is made to the OS 102. The OS 102 secures one of the four CPU cores that can be used by its own SMP function, and activates an application software thread or process (step S14).

  According to the above-described series of operation flows, the OS core of the PC 301 and the normal application 101 are recognized as if the CPU cores of the PC 301 and the PC 302 exist in an integrated physical casing, and a single hardware This makes it possible to implement parallel execution of multiple processes that could not be realized.

  Next, the case where the normal application 101 running on the PC 301 requests the OS to perform I / O (Input-Output) to the disk resource will be described with reference to the flowchart of FIG.

  When the normal application 101 requests I / O to the disk resource (step S21), the OS 102 makes an I / O request to the disk resource to the kernel 103. The kernel 103 issues a disk I / O request to the HAL 104, and the HAL 104 requests a disk I / O from the in-grid disk arbitration function 113 of the grid software 110 (step S22). The inter-grid disk arbitration function 113 performs access to the disk 127 of the PC 301 and the disk 227 of the PC 302 by a preset RAID method (step S23).

First, an example in which the disk RAID system is RAID 0 (mirroring) will be described.
The inter-grid disk arbitration function of the PC 301 simultaneously writes to the disks of both PCs via the inter-grid arbitration function of the PC 302, and reading is performed from the PC 301 disk (step S24). If a failure occurs in the disk of the PC 301, reading is performed from the disk of the PC 302 to enable continuous operation.

Next, an example in which the disk RAID system is RAID 5 (distributed parity) will be described.
When a write request occurs, the inter-grid disk arbitration function of the PC 301 divides the data and creates parity data. The divided data and parity data are written to the PC 301 and PC 302 disks. For data reading, the divided data is read from the PC 301 and the PC 302, and the data are combined and delivered to the OS and application (step S25). If a disk failure has occurred, use parity data to restore the data.
In the case of RAID 5, at least four disks are required.

  Through the above-described series of operation flows, the physical disk areas of the PC 301 and the PC 302 are recognized by the normal application of the PC 301 as if they were integrated, and it is possible to use a vast storage area that could not be realized with a single hardware. And

Next, the effect by embodiment mentioned above is demonstrated.
Conventionally, grid computing has been one of the means to build a low-priced and high-performance computing system, but it is necessary to develop dedicated application software in order to perform parallel processing efficiently. In addition, the memory space that can be used in one process is limited to the physical memory of one grid.

  On the other hand, according to the above-described embodiment, large-scale parallel processing is realized without modifying a commonly used application that supports multi-threading, and a large-capacity memory space, disk It became possible to secure space.

  More specifically, grid computing has traditionally been one of the methods for constructing a low-priced and high-performance computing system, but dedicated application software is required for efficient parallel processing. In addition, the memory space that can be used in one process is limited to the physical memory of one grid.

  In order to perform a large amount of numerical computation processing at high speed in the conventional grid environment, it is necessary to properly implement control related to arbitration between grids in the application, and very advanced application software development technology is required. It was. In addition, there is a restriction that application software (COTS (Commercial off-the-shelf) software) that is commercially available cannot benefit from grid computing.

  On the other hand, according to the above-described embodiment, the processing application can perform grid processing in a scalable manner by implementing very general multi-thread and multi-process processing.

  Also, by utilizing the inter-grid disk arbitration function, it has become possible to easily realize a large-capacity and highly available hard disk environment.

Each of the above-described embodiments is a preferred embodiment of the present invention, and the present invention is not limited to this, and various modifications can be made based on the technical idea of the present invention.
For example, in the above-described embodiment, it has been described that two PCs are connected as a configuration of the computer system. However, the configuration is not limited to this configuration, and how many computers are connected and perform processing by grid computing. May be.

  In the above-described embodiment, an example constructed by the hierarchical structure shown in FIG. 4 is taken as an example. However, if the grid software is provided in a hierarchy below the kernel and above the hardware, this hierarchical structure is used. For example, the structure shown in FIGS. 3 and 5 can be similarly applied according to the operating system to be used, the restrictions and characteristics of the kernel, and the like.

Further, by recording a processing procedure for realizing the computer system as each of the above-described embodiments on a recording medium as a program, the above-described functions according to each embodiment of the present invention are supplied from the recording medium. Thus, the processing can be realized by causing the CPU of the computer constituting the system to perform processing.
In this case, the present invention can be applied even when an information group including a program is supplied to the output device from the above recording medium or from an external recording medium via a network.
That is, the program code itself read from the recording medium realizes the novel function of the present invention, and the recording medium storing the program code and the signal read from the recording medium constitute the present invention. It will be.
As this recording medium, for example, flexible disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD + RW, magnetic tape, non-volatile A memory card, ROM or the like may be used.

  According to the program according to the present invention, each function in the above-described embodiment can be realized in a computer system controlled by the program and each computer constituting the system.

As described above, the present invention can be suitably applied to a computer system that requires a large amount of CPU resources, memory resources, and disk resources in large-capacity image data processing.
It is also beneficial to apply it to a system that focuses on securing disk resources and needs to secure large-capacity disk resources at a low price.

It is a figure which shows the outline of embodiment of this invention. 1 is a block diagram illustrating an example grid computing environment according to an embodiment of the invention. FIG. It is a block diagram which shows the structural example of the computer by one Embodiment of this invention. It is a block diagram which shows the structural example of the computer by one Embodiment of this invention. It is a block diagram which shows the structural example of the computer by one Embodiment of this invention. It is a block diagram which shows the structural example of the computer demonstrated as this embodiment. It is a block diagram which shows the example of a structure and operation | movement of a computer system as this embodiment. It is a flowchart which shows the memory allocation operation | movement by this computer system. It is a flowchart which shows CPU resource request | requirement operation | movement by this computer system. It is a flowchart which shows the access request | requirement operation | movement to the disk resource by this computer system. 1 is a block diagram illustrating an example of a general grid computing environment.

Explanation of symbols

101 Normal application (application)
102 OS
103 Kernel 104 HAL
110 Grid software 111 Inter-grid CPU arbitration function 112 Inter-grid memory arbitration function 113 Inter-grid disk arbitration function 120 Hardware 121 NIC
122 High speed NIC
123, 124 CPU core 126 Memory (physical memory)
127 disk (hard disk)

Claims (16)

A computer that is connected to other computers and can be processed by grid computing,
A computer comprising grid software for realizing the processing by the grid computing below a kernel in a software hierarchy and above hardware. The grid software includes at least an inter-grid memory arbitration function that allows a kernel to recognize a memory included in the computer and the other connected computer.
The inter-grid memory arbitration function secures a memory space in the computer when the physical memory of the computer has a sufficient margin in response to a memory allocation request, and does not have a sufficient margin in the physical memory of the computer 2. The computer according to claim 1, wherein a memory securing request is made to said other computer.   The inter-grid memory arbitration function, when a memory space is secured in the computer or the other connected computer, notifies the information on the secured memory space to higher-layer software. Item 3. The computer according to item 2.   The grid software includes at least an inter-grid CPU arbitration function that allows the kernel to recognize a CPU core included in the computer and the other connected computer when the kernel is started. The computer according to any one of 1 to 3. The grid software includes at least an inter-grid disk arbitration function that allows a kernel to recognize a hard disk included in the computer and the other connected computer.
The inter-grid disk arbitration function accesses each of the hard disks included in the computer and the other connected computers by a predetermined RAID method in response to a request for access to the hard disk. Item 5. The computer according to any one of Items 1 to 4.   A computer system comprising a plurality of computers according to any one of claims 1 to 5 connected to each other and capable of processing by grid computing. A distributed processing method in a computer system capable of processing by grid computing by connecting a plurality of computers,
A distributed processing method characterized in that when processing by an application is performed by the grid computing, grid software provided below a kernel and above hardware in a software hierarchy is used.   When a memory allocation request is issued from an application, if there is sufficient room in the physical memory of one computer, a memory space is secured in the computer, and there is not enough room in the physical memory of the one computer. 8. The distributed processing method according to claim 7, wherein a request for securing the memory is sent to another computer.   9. The distributed processing according to claim 8, wherein when a memory space is secured in response to a memory securing request by the application, information related to the secured memory space is notified from the grid software to upper layer software. Method.   10. The distributed processing method according to claim 7, wherein when the kernel is activated, the grid software causes the kernel to recognize the CPU core included in the computer system. 11.   11. The hard disk included in the computer system is accessed by a predetermined RAID method when an access request to the hard disk is issued from an application. Distributed processing method. A computer program connected to other computers and capable of processing by grid computing,
A program for causing a computer to execute processing by an application in the grid computing by using grid software provided below a kernel and above hardware in a software hierarchy.   When a memory allocation request is issued from an application, if there is sufficient room in the physical memory of the computer, a memory space is secured in the computer, and if there is not enough room in the physical memory of the computer, the other 13. The program according to claim 12, which causes the computer to execute a process of making a memory allocation request to the computer.   When a memory space is secured in response to a memory securing request by the application, the computer is caused to execute a process of notifying the information related to the secured memory space from the grid software to the upper layer software. The program according to claim 13.   13. The system according to claim 12, wherein when the kernel is started, the computer is caused to execute processing for causing the grid software to recognize the CPU core included in the computer and the other connected computer. 14. The program according to any one of 14.   16. The computer according to claim 12, wherein when a request to access a hard disk is issued from an application, the computer is caused to execute a process of accessing each hard disk included in the computer system by a predetermined RAID method. The program according to any one of the above items.

Images