JP3628782B2 - Parallel distributed processing system - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a parallel and distributed processing system in which a plurality of processors advance processing in cooperation.
[0002]
The current parallel and distributed processing environment has become the monopoly of some experts due to its own complexity and programming difficulties for parallel processing, but a single CPU processing bottleneck. At present, there is an urgent need to provide an environment that enables easy and high-performance parallel processing for general computer users.
[0003]
[Problems to be solved by the invention]
A parallel processing distributed system in which a plurality of processors cooperate in processing, particularly a system environment in which one task is processed in parallel and distributed while coordinating multiple heterogeneous processors as one cluster in a LAN or WAN environment. Yes. The problem in providing such a parallel processing environment to general users is simplicity and ease of learning, but this creates a trade-off relationship with performance.
[0004]
In the state of the art, it is necessary to sacrifice a considerable degree of simplicity for performance, or to accept a significant decrease in performance for simplicity.
In terms of performance, the delay and throughput of data transfer between processors becomes a problem, but this can essentially be reduced to the problem of delay in data transfer. This is because in a calculation with a high degree of parallelism, the unit of data to be transferred is small, and the delay depends only on the number of transfers that is not related to the transfer amount. In order to reduce the delay that depends on the number of transfers as a whole, it is necessary to accumulate some data to be transferred in a buffer and transfer it all at once. The fact is that the factors are intricately intertwined, so it's hard to see that they don't actually experiment.
[0005]
In order to execute a program in parallel, it must be divided into units for parallel execution. However, if you divide into smaller units, more CPUs can be used, but instead the overhead of synchronization due to smaller execution units, overhead due to increased context switches, data delays, and increased data transfer volume , Resulting in increased paging due to memory fragmentation, and again, there is a trade-off.
[0006]
It is desirable for end users to enjoy the benefits of faster and larger programs through parallel processing. At present, however, end users are also required to be able to obtain a certain level of performance. We face the contradiction that the user has to acquire the complicated know-how of parallel processing and tune the program.
[0007]
Conventionally, high-level languages for parallel processing, for example, Occam (A. Burns, PROGRAMMING IN occam 2, Addison-Wesley, 1988), HPF (High Performance Fortran Forum, High Performance), are used as means for solving these problems. FORTRAN LANGUAGE SPECIFICATION, 1994), as a function type, CLEAN (R.Plasmeijer and M.van Ekelen, Functional Programming and Parallel Graph Rewriting, LO type, PA). High-level languages such as ARLOG, Addison-Wesley, 1989) have been developed.
[0008]
Further, as a high-level library interface, for example, PVM (A. Geist, A. Beguelin, J. Dongarra, W. Jiang, R. Manchek and V. Sunderam, PVM: Parallel Virtual Machine-A User's User's User's Guide. Networked Parallel Computing-, MIT press, 1994), MPI (Message Passing Interface Forum, MPI: A Message-Passing Interface Standard, May 5, 1994) has been developed.
[0009]
However, high-level languages do not have sufficient performance, or expert-level know-how is necessary to achieve performance, and high-level libraries have not yet reached a level that can be used by end users.
[0010]
As another intermediate solution, a method of combining a relatively low-level procedural sequential language and an instruction language for parallel processing (I. Foster, R. Olson and S. Tuecke, Productive Parallel Programming, Scientific Programming, Vol. 1, pp. 51-66, 1992; LA Crown and TJ LeBlanc, Parallel Programming with Control Abstraction, ACM Transactions on Programming Language 3, p. 576, 1994).
[0011]
These methods are not end users in all aspects, but are experts in sequential processing, but with respect to parallel processing, those who are relatively close to end users are interested in low-level sequential language tuning and parallel processing. Tuning is separated and a simple and uniform tuning style is introduced only to the parallel processing interface.
[0012]
The system targeted by the present invention is based on the latter concept, but promotes further uniformization of the parallel processing interface part and versatility for expert users, and is flexible in the interface with the sequential processing part. In order to have the nature, we try to unify the whole with the semantics of procedural languages.
[0013]
For example, when developing a parallel processing program on a workstation group on a network or a dedicated multi-CPU parallel computer, the parallel computer is usually constructed using only one workstation or on one workstation. Start by developing a program using the simulator. In many cases, routines for programs that have been developed in the past are made into a library. If an available library already exists, program development is started from the point where it is reused.
[0014]
In this way, program development progresses, and at the stage of actually executing the program and debugging the entire program, conventionally, binary code that can be created by linking an existing library to the newly developed routine is executed. Sometimes it was repeated every time to all workstations or all CPUs. This method has a problem of reducing the efficiency of program development because the library portion that does not require debugging is transferred every time it is debugged. Further, even in the stage where the program is actually executed and operated, the library portion is similarly transferred each time, which causes a long time for starting the program.
[0015]
In particular, when processing is performed using a group of workstations on the network, there is a demand for using different types of workstations. In this case, one approach is to create a partial program for sequential processing in units of functions using, for example, the above-described parallel processing message passing interfaces such as PVM and MPI, and connect them in parallel to perform parallel processing. is there. In this way, the native code of each program can be different for each type of workstation. However, for each type of workstation, there is a disadvantage that the program must be compiled to create native code, which makes the work complicated.
[0016]
Another approach is to set up virtual code and place the interpreter on each workstation for parallel processing. Since this method uses only one type of code, there is an advantage that compilation and other operations are simple, but there is a drawback that overhead for interpreting at the time of execution of virtual code is large.
[0017]
As a method for overcoming this drawback, there is a method in which an interface for calling native code from virtual code is provided, and the overhead of the interpretation is reduced by executing a portion of the native code. However, in this method, a standard native code library is linked in advance to the interpreter, and general native code created by the user cannot be linked and used.
[0018]
The present invention solves the above-described problems, provides an environment in which end users can easily describe parallel and distributed computations, and enables flexible native code usage from virtual code, thereby enabling high-performance program execution. The purpose is to realize the environment.
[0019]
[Means for Solving the Problems]
FIG. 1 is an explanatory diagram of the principle of the parallel distributed processing system of the present invention.
In FIG. 1, processors 10 and 10 ′ are virtual code execution means 11 and 11 ′, library tables 13 and 13 ′, dynamic link means 14 and 14 ′, native code libraries 15 and 15 ′, and a scheduler 16, respectively. , 16 ′. The programs 12 and 12 ′ operating on the processors 10 and 10 ′ are virtual codes described in a format (syntax) common to the processors 10 and 10 ′ and instruction code groups specific to the processors 10 and 10 ′. It consists of written native code.
[0020]
In the present invention, a user program in which a plurality of processors 10 and 10 ′ advance one calculation in cooperation with each other on the network 20, virtual codes described in a format common to the processors 10 and 10 ′, It is separated into native code libraries 15 and 15 'composed of native codes described in instruction codes specific to the processors 10 and 10', and the native code libraries 15 and 15 'required for executing virtual code Are dynamically linked to the virtual code by the dynamic link means 14, 14 'in each processor 10, 10'. The native code libraries 15 and 15 'are stored in local file systems, respectively.
[0021]
The virtual code execution means 11 and 11 ′ are means for interpreting and executing the virtual codes of the programs 12 and 12 ′, respectively. When a call to native code that is not yet mapped in the program 12, 12 'is detected during execution of the virtual code, the native code library 15, 15' is mapped by the dynamic link means 14, 14 '. , The native code is executed by the native code interface in the virtual code execution means 11, 11 ′.
[0022]
The library tables 13 and 13 ′ include the target native code library 15 and the dynamic code when the dynamic link means 14 and 14 ′ dynamically link the native code libraries 15 and 15 ′. 15 is a table for obtaining storage location information of 15 ′.
[0023]
The schedulers 16 and 16 'are means for performing control to give a CPU execution right to a process (or thread) that is a unit for executing the programs 12 and 12', and perform virtual code execution means while communicating with other processors. 11, 11 'is controlled. In addition, it has a function of migration (movement) for sending and receiving processes (or threads) to and from other processors for load balancing and the like.
[0024]
For example, when the scheduler 16 of the processor 10 migrates a process (or thread) to another processor 10 ′, the scheduler 16 stores information necessary for dynamic linking with the virtual code portion, for example, the library table 13. Send, do not send the native code part. In the migration destination processor 10 ′, the received virtual code is executed from the top execution start point or execution stop point by the virtual code execution means 11 ′, and when the native code call is detected in the execution of the virtual code, The native code library 15 ′ is mapped to the program 12 ′ by the link means 14 ′ and the execution proceeds.
[0025]
Instead of dynamically linking the native code libraries 15 and 15 'when the native code call is detected in the virtual code execution by the virtual code execution means 11 and 11', the scheduler 16 and 16 ' When the process (or thread) is migrated and the scheduler 16, 16 'receives information necessary for dynamic linking with the virtual code from other processors, the native code library 15, You may make it link 15 'dynamically.
[0026]
DETAILED DESCRIPTION OF THE INVENTION
In the parallel distributed processing system according to the present invention, in order to provide a high-performance execution environment that is easy to parallelize, for example, on a networked heterogeneous cluster, any process being executed is appropriately migrated (moved). ), And it is possible to link the processes on the network with high-speed stream communication.
[0027]
In other words, by assigning the CPU to the processing of another process while a certain process is waiting for a communication response, the CPU utilization efficiency is improved, and communication by execution of other processes is also overlapped in time. It is possible to provide a system that can reduce communication latency when averaged as a whole.
[0028]
As an embodiment of the present invention, an interpreter for parallel computation is started on a plurality of workstations of different types on the network, and information necessary for a virtual code and a dynamic link is sent between them. The case of parallel calculation using the library of will be described.
[0029]
A process corresponding to a master processor and a slave processor is realized as a process in a specific system, and a plurality of virtual code interpreters are executed while being scheduled in this process. This process allows input / output data between workstations. , Virtual code, library table, interpreter control information, etc.
[0030]
(1) Request Drive Method FIG. 2 is a diagram illustrating an implementation example using a request drive method according to an embodiment of the present invention. In FIG. 2, a processor 30 includes an interpreter 31 having a dynamic linker 32 and a native code interface 33, a program 34 written in virtual code, and a library table for storing and looking up dynamic link information. 35, a native code library 36 stored in the local file system, and a scheduler 37 for controlling the interpreter 31 while communicating with other processors. The interpreter 31 corresponds to the virtual code execution means 11 and the dynamic link means 14 shown in FIG. The configuration of the processor 30 ′ is the same.
[0031]
Here, it is assumed that an event for moving the process from the processor 30 to the processor 30 ′ occurs. This event may be caused by an explicit movement instruction from the user, or may be autonomously generated by the scheduler 37 by load balancing control. The scheduler 37 of the processor 30 transfers the virtual code and the library table 35 in the program 34 to the processor 30 ′. The scheduler 37 ′ of the processor 30 ′ receives the above information from the processor 30 and sets it as a program 34 ′ and a library table 35 ′ on the memory.
[0032]
Here, the call instruction of the native code is “lib_call“ func ”” when the function name to be called is “func”. The file name for storing the library including the function with the function name func is assumed to be “/opt/X11/lib/libX11.so” here.
[0033]
The interpreter 31 ′ executes the virtual code of the program 34 ′, detects the call instruction “lib_call“ func ”” of the native code, and extracts the function name func to be called. If this is registered in a hash table described later, the function having the function name func is directly executed by using the native code interface 33 ′. If it is not registered in the hash table, the dynamic linker 32 ′ is started, and the local name “/opt/X11/lib/libX11.so” is determined from the library name registered in the library table 35 ′. The contents of the corresponding native code library 36 'in the file are accessed, a function name hash table is constructed, and this is combined with the library name of the library table 35'.
[0034]
The construction of the hash table of the library is continued until the function with the function name func is found. When the func is found, the library including the func is mapped to the storage area of the native code in the program 34 'and used by the library. Link-edit with other standard libraries, register the entry points of each function in the hash table, and make each function in the library executable.
[0035]
Finally, using the native code interface 33 ', the entry point of the library function corresponding to func is searched and the function is executed.
FIG. 3 is a process flowchart of the request driving method according to the configuration example of FIG.
[0036]
Assume that the processor 30 shown in FIG. 2 is a master CPU and the processor 30 ′ is a slave CPU. The processor 30 'starts processing when there is no process (thread) to process anything at the beginning, and there is a process (thread) migration from the master processor 30 in step S11 of FIG. .
[0037]
First, in step S12, the virtual code and the library table are received from the master processor 30. In step S13, the received virtual code is executed by the interpreter 31 ′. As a result of execution, if a native code library call instruction is detected in step S14, the process proceeds to step S15, and if an exit instruction instructing the end of execution is detected, the process is stopped. If it is another instruction, steps S13 to S14 are repeated until either the native code library call instruction or the exit instruction is detected.
[0038]
In step S15, the interpreter 31 ′ determines whether or not the corresponding native code library 36 ′ is mapped based on the hash table of function names. If the native code library 36 'is not mapped, the process of step S16 is performed. If the native code library 36' is mapped, the process proceeds to step S17.
[0039]
In step S16, the file name of the native code library 36 'to be mapped is obtained from the library table 35', the native code library 36 'is mapped by the dynamic linker 32', and link editing is performed. .
[0040]
Thereafter, in step S17, control is passed to the native code interface 33 ′ of the interpreter 31 ′ to execute the native code. When the execution of the native code is completed, the process returns to step S13, and the execution of the virtual code is continued in the same manner.
[0041]
(2) Pre-processing method FIG. 4 is a diagram showing an implementation example of the pre-processing method according to an embodiment of the present invention. The implementation example shown in FIG. 4 has substantially the same configuration as the implementation example shown in FIG. 2, except that the dynamic linkers 54 and 54 ′ are not in the interpreters 51 and 51 ′ but in the schedulers 53 and 53 ′. Is different.
[0042]
The procedure up to the point where the virtual code and the library table are transferred from the master processor 50 and received by the slave processor 50 ′ is the same as the above-described request driving method. However, in this system, after the slave processor 50 ′ receives the virtual code and the library table from the master processor 50, before starting the interpreter 51 ′, all the libraries in the received library table 56 ′ are processed. A function name hash table is created in advance, the library is mapped to the native code storage area of program 55 ', link-edited with the standard library, etc., and the library function entry point is registered in the hash table. Do it all together first.
[0043]
When the interpreter 51 ′ executes the virtual code of the program 55 ′ and detects the call instruction “lib_call“ func ”” of the native code, the interpreter 51 ′ starts the native code interface 52 ′ directly and is generated in the native code. The function func is executed.
[0044]
FIG. 5 is a process flowchart of the preprocessing method according to the configuration example shown in FIG.
In the situation where the slave processor 50 'shown in FIG. 4 does not have any process (thread) to process anything at first, there is a process (thread) migration from the master processor 50 in step S21 of FIG. And start processing. In step S22, the processor 50 ′ receives the virtual code and the library table from the master processor 50.
[0045]
Next, in step S23, all the native code libraries 57 'registered in the received library table 56' are mapped by the dynamic linker 54 'in the scheduler 53', and link editing is performed.
[0046]
In step S24, the virtual code is executed by the interpreter 51 '. As a result of execution, if a native code library call instruction is detected in step S25, the process proceeds to step S26, and if an exit instruction instructing the end of execution is detected, the process is stopped. If it is another instruction, steps S24 to S25 are repeated until either the native code library call instruction or the exit instruction is detected.
[0047]
In step S26, the native code is executed by passing processing to the native code interface 52 'of the interpreter 51'. When the execution of the native code is completed, the process returns to step S24 and the execution of the virtual code is continued in the same manner.
[0048]
Depending on the input data, etc., various libraries are used during execution. If the number of libraries used during execution is small for a large number of registered libraries, the request drive method shown in FIG. 2 is more advantageous. is there.
[0049]
On the other hand, if most libraries registered at runtime are used for the processor, the preprocessing method shown in Fig. 4 should check whether the library is already mapped by each native code call instruction. There is no advantage.
[0050]
【The invention's effect】
According to the present invention, there is provided a mechanism for separating virtual code as intermediate code and native code specific to each processor and dynamically linking native code when executing virtual code or receiving virtual code. As a result, it is possible to provide a simple parallel distributed processing system that maintains the high processing speed of parallel processing among a plurality of heterogeneous processors and that is also suitable for use by end users.
[0051]
For example, when developing a parallel processing program on a workstation group on a network or a dedicated multi-CPU parallel computer, a system standard library, a user-created library that has already been developed elsewhere, a user-created library that has been debugged, etc. There is no need to update the library.
[0052]
In addition, since native code specific to each processor does not need to be transferred from the master processor every time or prior to execution, debugging and other development work are reduced, and development is accelerated. Also, when the development is finished and the program is actually activated and used, the program activation time is shortened.
[0053]
Furthermore, when changing a program, it is possible to flexibly absorb the changes of the entire program by changing only the virtual code part, and to draw out performance by using the native code library. The effect is that it becomes easy.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating the principle of the present invention.
FIG. 2 is a diagram illustrating an implementation example using a request driving method according to an embodiment of the present invention.
FIG. 3 is a process flowchart according to a request driving method;
FIG. 4 is a diagram illustrating an implementation example using a preprocessing method according to an embodiment of the present invention.
FIG. 5 is a process flowchart according to a pre-processing method.
[Explanation of symbols]
10, 10 'processor 11, 11' virtual code executing means 12, 12 'program 13, 13' library table 14, 14 'dynamic linking means 15, 15' native code library 16, 16 'scheduler 20 network

Claims (2)

In a parallel and distributed processing system in which multiple processors work together,
The processor is
Library storage means for storing a library of native codes described in instruction codes specific to each processor;
Virtual code execution means for interpreting and executing virtual code that is a program described in a format common to each processor;
A dynamic linking means for dynamically linking a library of native codes stored in the library storage means when a call to native code is detected in the execution of virtual code;
Means for transmitting virtual code to be executed to another processor and information about the storage location of the virtual code to be executed and the target native code library required for dynamic linking to the other processor; A parallel distributed processing system characterized by comprising: In a parallel and distributed processing system in which multiple processors work together,
The processor is
Library storage means for storing a library of native codes described in instruction codes specific to each processor;
Virtual code execution means for interpreting and executing virtual code that is a program described in a format common to each processor;
Means for transmitting virtual code to be executed to another processor and the storage location information of the virtual code to be executed and the native code library to be necessary for dynamic linking to the other processor;
Dynamic link means for dynamically linking libraries of native codes stored in the library storage means when receiving virtual code from another processor and executing the virtual code Parallel distributed processing system characterized by

Images