JPH09146904A - Address space sharing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To effectively use a local memory space by evading the problem of an address conflict which is caused when an arbitrary thread (process) which is being executed is moved to another processor in the address space sharing space of a parallel decentralized processing system, etc., wherein plural processors carry forward processing collaboratively. SOLUTION: The address spaces of processors 11 and 11' are divided into spaces that plural threads share and spaces, such as a stack, that the threads do not share. The processors 1 and 1' are provided with CPU address/logical address converting circuits 12 and 12' and for access to the unshared space of the latter, the value of a base register 15 is added to its access request address to perform address conversion, making the address space such as a stack relative and moving it optionally in parallel.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an address space sharing system in which a plurality of processors or threads operate by sharing an address space in a parallel distributed processing system in which a plurality of processors cooperate in processing.

The current parallel distributed processing environment has become the monopoly of some experts due to its own complexity and the difficulty of programming for parallel processing, but processing by a single CPU Now that the bottleneck of is becoming apparent, there is an urgent need to provide an environment that enables general computer users to easily perform high-performance parallel processing.

[0003]

A parallel processing distributed system in which a plurality of processors cooperate to carry out processing, in particular, LA
A system environment has been considered in which a plurality of heterogeneous processors in an N, WAN environment are coordinated as one cluster to perform one task in parallel and distributed processing.
A problem in providing such a parallel processing environment to general users is simplicity and easiness of learning, but this causes a trade-off relationship between performance and performance.

In the state of the art, simplicity must be sacrificed for performance to a large extent, or performance must be significantly reduced for simplicity. In terms of performance, the delay and throughput of data transfer between processors becomes a problem, but this can be essentially reduced to the problem of delay in data transfer. This is because, in a calculation with a high degree of parallelism, the unit of data to be transferred is small, and the delay mainly depends on the number of transfers regardless of the transfer amount. In order to reduce the delay depending on the number of transfers as a whole, it is necessary to store the data to be transferred in a buffer to some extent and transfer them all at once. What is the optimum size of this buffer? What is there is that the factors are intricately intertwined,
In fact, I don't know what I didn't experiment with.

In order to execute a program in parallel, it must be divided into units for parallel execution.
However, if it is divided into smaller units, more C
Instead of making the PU available, it causes synchronization overhead due to smaller execution units, overhead due to increased context switches, data delay, increased data transfer amount, increased paging due to memory fragmentation, etc. And again there is a trade-off.

It is desired that end users can enjoy the advantages of speeding up and increasing the scale of programs by parallel processing. However, under the present circumstances, in order to obtain a certain level of performance, the end users are required. Also an expert
We face the contradiction of having to tune the program by acquiring the user's complicated know-how of parallel processing.

As a means for solving these problems, conventionally, a high-level language for parallel processing, for example, as a procedural type, Occ
am (A. Burns, PROGRAMMING IN occam 2, Addison-Wes
ley, 1988), HPF (High Performance Fortran Foru)
m, High Performance Fortran Language Specification,
1994), CLEAN (R. Plasmeijer and
M. van Eekelen, Functional Programming and Paralle
l Graph Rewriting, Addison-Wesley, 1993), as a logical type, PARLOG (T. Conlon, Programming in PARLO
Higher level languages such as G, Addison-Wesley, 1989) have been developed.

As a high-level library interface, for example, PVM (A.Geist, A.Beguelin, J.Don
garra, W.Jiang, R.Manchek and V.Sunderam, PVM: Parall
el Virtual Machine-A Users' Guide and Tutorial f
or Networked Parallel Computing-, MIT press, 1994
), MPI (Message Passing Interface Forum, MPI:
A Message-Passing Interface Standard, May 5,199
4) is being developed.

However, a high-level language does not have sufficient performance, or expert-level know-how is required to obtain the performance, and the high-level library has not yet reached a level usable by the end user. .

As another intermediate solution, a method of combining a relatively low-level procedural sequential language and an instruction language for parallel processing (I. Foster, R. Olson and S. Tueck
e, Productive Parallel Programming, Scientific Pro
gramming, Vol.1, pp.51-66, 1992; LACrowl and TJ
LeBlanc, Parallel Programming with Control Abstrac
tion, ACM Transactions on Programming Languages an
d Systems, Vol.16, No.3, pp.524-576, 1994) have been proposed.

These methods are not end users in all aspects, but those who are experts in serial processing but are relatively close to the end users in terms of parallel processing. It separates the tuning of parallel processing and introduces a simple and uniform tuning style only to the interface part of parallel processing.

The system targeted by the present invention is based on the latter idea, but promotes further uniformization of the parallel processing interface part and versatility for expert users, and In order to make the interface flexible, we are trying to unify the overall semantics of the procedural language.

For example, when a parallel program is executed on a group of workstations on a network or a dedicated multi-CPU parallel computer under the above parallel distributed processing system, a shared memory mechanism (virtual shared memory) is used. Mechanism, including the following).
A method of configuring an address space (real address space or logical address space) shared by U is known as a method that enables parallel programming that is the closest to programming using a single CPU. Further, since the address space such as the stack and various buffers does not need to be shared with other CPUs, instead of sharing all the address space, a part of the address space should be local to each CPU. There is also a way to do it.

The processing models of these methods are such that calculation subjects called a plurality of threads proceed in parallel with each other in a shared address space, sometimes in synchronization with each other. Each thread uses its own stack and advances the processing by exclusively using the CPU on the address space shared with other threads. That is, a thread reserves its own data area or temporary data on the stack and uses it for CPU operation, or when giving up the CPU usage right to another thread, the CP
By saving the contents of the U register on its own stack and then returning the saved contents of the register when the CPU usage right is handed over to itself, the consistency of the CPU state for each thread Retains sex.

As described above, although the stack space is a collection of dedicated spaces for each thread, if it is shared by all CPUs, the threads of one CPU are used by other CPUs, so that the other CPUs have their own. There is a problem that a lot of address space of the memory that cannot be used is created.

Also in the case of the method of placing the stack in the local address space, the thread can be moved because the stack must be placed in the same address space as before when the thread is moved to another CPU. In that case, the stack has to be practically placed at the (virtual) shared address, resulting in poor utilization efficiency of the address space.

FIG. 5 is a diagram for explaining the problems of the above-mentioned conventional technique. As shown in FIG. 5A, when the thread A that has been executed by the CPUi is moved to another CPUj and the execution of the thread A is continued on the CPUj, even in the address space of the CPUj, the address space of the CPUi It is necessary to arrange the stack at the same position as the stack for thread A of. However, if another thread C stack is already in that position, an address conflict occurs, and the thread cannot be moved.

To make the thread A movable, for example, as shown in FIG. 5B, an area is reserved in advance so that stacks for all threads do not overlap the address space of the stack. The need arises, and the important resource of the address space is wasted.

The present invention solves the above problems and prevents address collisions such as stacks in the shared memory system even when threads or processes move, thereby enabling effective use of the address space. The purpose is to do.

[0020]

According to the present invention, an address space is divided into a shared space and a local address space which is not shared by other CPUs, and a stack space dedicated to a stack address is formed in the local address space. Address conversion is not performed for memory access from the CPU to addresses other than the stack space, and only for access to the memory at the address of the stack space, the address is automatically converted to an offset from the origin, The contents of a dedicated register (stack base register: sbr) are added together.

FIG. 1 is a diagram showing a configuration example of the present invention.
For example, as shown in FIG. 1 (A), a plurality of threads 13a, which are the subject of calculation, who obtain the processor usage right on the plurality of processors 11 and 11 'connected by the network 10.
In the system in which 13d operates, each processor 1
The memory address spaces 1 and 11 'are divided into a first address space shared by a plurality of threads and a second address space not shared by threads. In addition,
Here, the calculation subject that obtains the processor usage right is described as a thread, but the present invention can be similarly applied even if this calculation subject is a process.

Each of the processors 11 and 11 'has an address designated by a thread when accessing a memory (this is a C address).
CPU address / logical address conversion circuits 12 and 12 'for converting a part of a PU address into another address (this is called a logical address) are provided.

CPU address / logical address conversion circuit 1
Reference numerals 2 and 12 'are circuits composed of a space discriminating circuit 14, a base register 15, an adding circuit 16 and an address register 17, as shown in FIG.

The space discriminating circuit 14 determines whether a memory access request from a thread is directed to a shared address space (for example, a space other than the stack space) in the address space or an address space which is not shared by the threads (for example, the stack space). ) Is a circuit for determining whether or not it is.

The base register 15 is a base register for making a space that is not shared between threads a relative address space.
A means for holding an address, for example, a stack base address which is an offset value of a stack for each thread is held. The adder circuit 16 is a circuit that adds the base address held in the base register 15 to the address of the access request when the access request is for a space such as a stack space that is not shared by threads.

The address register 17 is provided for the adder circuit 1 when the CPU address indicates a non-shared space such as a stack space.
If an address obtained by adding the base address in 6 is held, and if the CPU address indicates a shared space, its CP
The U address is held as it is and output as a logical address.

With this CPU address / logical address conversion circuit 12, 12 ', as shown in FIG.
The position of the stack space of each thread in the PU address space is based on the origin of the address in the logical address space.
It is possible to move to the position where the value sbr of the register 15 is added, and it is possible to avoid address collision.

[0028]

BEST MODE FOR CARRYING OUT THE INVENTION In order to provide a high-performance execution environment that can be easily parallelized, this system can appropriately execute arbitrary threads (or processes) on a networked heterogeneous cluster. It realizes that the entire process can be carried out while moving (migrating) to, and that the processes on the network are connected by high-speed stream communication.

That is, by allocating the CPU to the processing of another process while a certain process is waiting for a communication response, the utilization efficiency of the CPU is improved and the communication by the execution of the other process also overlaps in time. Therefore, it is intended to provide a system capable of reducing the communication latency when averaged as a whole.

In order to flexibly deal with such a heterogeneous computer environment, a virtual code (intermediate code) method that does not depend on a computer is adopted, and further, thread (process) migration at any time is possible. In order to do so, a virtual shared memory mechanism is adopted.

(1) Virtual code system In this system, a plurality of virtual codes and interpreters of the virtual codes are distributed and arranged on the network, and access to the remote memory on the virtual shared memory is converted into high-speed communication. While proceeding with the process.

The virtual code can be transferred between the servers when necessary, but the virtual code can be held in each server in advance so that the transfer is not required for each movement.
The access to the heap area of another CPU is trapped every time the process is executed, and the data is transferred in cell units each time. Therefore, in this case, it is not necessary to transfer the virtual code and the data in the heap area when the thread moves, and only the stack area of the process and the contents of several registers used by the virtual code interpreter need to be transferred.

(2) Virtual shared memory If the stack area is also managed by the mechanism of the virtual shared memory, it is not necessary to transfer it at the time of movement, but the stack can be accessed only by the process that uses it to advance the processing. Therefore, if virtual shared memory is used, the memory space is reserved on all servers, and the amount of local memory available on all servers becomes small. In addition, there is also the overhead of virtual shared memory, which reduces efficiency.

In order to enable dynamic movement during the execution of the process, it is necessary to make the virtual code position-independent on the network. For that purpose, it is necessary to virtually share all memory addresses appearing in the virtual code with all computers.
In order to support at the operating system (OS) level, virtual shared memory with a large-scale mechanism (D.Lenoski, J.Laudon, K.Gharachorloo, A.Gupta, J.Henn
essy, The Directory-Based Cache Coherent Protocol
for the DASH multiprocessor, IEEE Proceedings of t
he 17th Annual International Symposium on Computer
Architecture, pp.148-159, 1990).

In this system, in order to move the stack between servers at an arbitrary time, the virtual stack address space is set to the stack base register (sb) which is an internal register of the virtual code when the virtual code is designed.
Relative addressing using r). In other words, on the actual computer, the contents of the stack are copied to another address at any time, and the start of that address is set in the stack base register so that execution can continue without conflict.

That is, the entire virtual code area may be copied on demand at once into the memory space reserved immediately before the first process starts on each computer. The memory space and heap area with other writes are
Cells are managed in units of variable-length write-once memory, and a virtual shared memory is constructed only for access to these limited cells.

Therefore, since the memory space internally used by the interpreter, the stack space of the process, the stream buffer, etc. can all be stored on the local computer, the cell space which is the target to be handled by the virtual shared memory. Is quite small. Moreover, since the cell is write-once, the mechanism for maintaining the coherence of the cache of cell data can be omitted, and the cache of the created cell is treated as permanent local data without the cost of the mechanism for maintaining the coherence of the cache. Can be held. As a result, the overhead due to using the virtual shared memory mechanism can be reduced.

FIG. 2 is a diagram for explaining the process of converting the CPU address space into a logical address used by the virtual memory mechanism. In FIG. 2, the CPU other than the stack space
A which is the address x is A ′ of the logical address x as it is
B, which is the CPU address of the stack space
Is once converted to the offset y from the origin, then added to the value of the stack base register (sbr), and converted to the logical address (sbr + y).

As described above, since the CPU / logical address conversion is implemented at the CPU level, an algorithm or circuit optimized for the address conversion can be built in advance, and the address conversion is explicitly performed at the program level. Compared with the case, it is very fast and the overhead due to this is very small.

Moreover, as the number of CPUs increases by paying the cost of this small overhead,
It is possible to ensure scalability that the address space that can be effectively used by the CPU can be enlarged as a whole.

[0041]

FIG. 3 is a diagram showing an embodiment of the present invention. A language for making parallel calculations by a plurality of workstations (processors 30, 30 ', ...) On the network is created, and the stack base of this method is used for the interpreter.
We designed virtual code with registers and realized it. The language is designed so that any thread can move to another processor at any time.

The thread schedulers 33, 33 'of the operating system (OS) are means for performing execution control of each thread and movement control between processors while communicating between the processors. Thread control table 3 managed by the thread scheduler 33, 33 '
4, 34 ′ includes, as control information related to the execution of each thread, for example, a thread ID for identifying the thread, a thread execution time, a thread execution right, a processor (CPU) number to which the thread is assigned, and when Conditions for migrating threads to, the value of stack base register (sbr),
The stack pointer (sp) value and the like are stored.
The thread control table 34, 34 'is the processor 3
When 0, 30 'schedule execution of threads, when transferring threads to other processors,
Registered, updated, and deleted.

FIG. 3 shows thread A of processor 30 moving to processor 30 '. Here, the thread A uses the thread stack A of the address space 31 on the processor 30, but in the address space 31 ′ of the destination processor 30 ′, the thread C already has the address of the thread stack A. Since it is used, the thread stack A cannot be moved by the conventional method.

However, in the case of this system, the thread A on the processor 30 has the value of the stack base register called sbr, and the address space dedicated to the stack is made relative by this. Therefore, thread A
Sb so that the value of the stack base register on the processor 30 ′ does not overlap the thread stack C when the processor is moved from the processor 30 to the processor 30 ′.
By changing to r ′, thread A becomes processor 3
Thread stack A can be used on 0'without problems.

As described above, it is only necessary to change the value of the stack base register at the time of address collision, because all accesses to addresses in the stack space of the processor are s.
This is because it is a relative value regarding br. Further, since the address space used on the processor and the address space (logical address space) of the virtual memory are indirectly associated with each other through the address conversion using sbr, the load and store on the processor are performed. Address,
This is because all the addresses appearing in and other calculations are the CPU address of the processor.

That is, the stack base register (s
In a relative stack address space using (br), the location of the stack can be freely translated in the logical address space before and after the move without making the access request source aware. The stack of a moving thread can be copied to any free stack space without negotiation.

As described above, the thread A is processed by the processor 3
When moving from 0 to the processor 30 ′, the thread scheduler 33 does not determine whether the movement of the thread A causes an address collision in the address space 31 ′ of the destination processor 30 ′, and the thread A It is possible to transfer the thread A to the thread scheduler 33 'of the processor 30' to move the thread A.

Destination thread scheduler 33 '
Allocates thread stack A to an empty stack space in address space 31 ',
Sbr ′, which is a relative value from the origin of the above, is set in the thread control information of the thread A, and the processor usage right is given to the thread A at the timing of the execution schedule to execute the thread A.

Here, the stack pointer (sp)
Holds a relative value from the stack base register (sbr), and the value is held even after the thread is moved.
In the present embodiment, as a method for determining whether the received CPU address is the stack space or the other space in the space determination circuit 14 in the CPU address / logical address conversion circuit shown in FIG. , CP
If the upper 2 bits of the U address is “11”, it is determined to be the stack space, and if the upper 2 bits are other than “11”, it is determined to be the space other than the stack space. Value of base register 15 (sb
When r) is added, the upper 2 bits of the CPU address are masked and the upper 2 bits are set to “00” before adding.

The space may be determined not by using a part of the CPU address but by using a control signal or the like obtained from an instruction code, for example. FIG.
3 is a process flowchart centering on the process of the thread scheduler shown in FIG. This processing flow chart shows the processing flow of the OS and each thread in each CPU (processor), and receives threads from other CPUs (OS) (steps S1 to S4) and other CPUs (O).
Send thread to S (steps S5 to S8), own C
Execution control of threads in PU (steps S9 to S15)
It consists of three loops.

In step S1, it is determined whether or not there is a thread transfer request from another CPU (OS). If there is a transfer request, the process of step S2 is performed, and if there is no transfer request, the process proceeds to step S5.

In step S2, the thread stack is received from another CPU (OS) and its contents are copied to a free area in the address space. In step S3, the head of the address space where the thread stack is copied is set to the save area of the stack base register sbr for that stack. The change of the sbr value of the moved thread is executed by writing to the sbr area on the stack control table reserved for the stack by the CPU.

In step S4, the control information of the thread using the stack transferred from another CPU (OS) is received and set in the thread control table. Then, it returns to step S1.

At step S5, for each thread,
Check the movement conditions in the thread control table,
Determine which thread to move. In step S6, it is determined whether or not there is a thread that satisfies the movement condition,
If it exists, the process of step S7 is performed, and if it does not exist, the process proceeds to step S9.

In step S7, the thread stack of the thread that matches the movement condition is transferred to another CPU (OS). In step S8, the control information of the thread of the stack transferred to another CPU (OS) is transferred from the thread control table, and the information in the thread control table is erased. Then, it returns to step S5.

In step S9, the thread to be executed is determined. In step S10, it is determined whether or not there is a thread to execute. Step S if it exists
The process of 11 is performed, and when there is no thread in an executable state, the process returns to step S1.

In step S11, the stack base
The value of the register (sbr) is obtained from the stack control table of the stack to be executed and set in the stack base register.

In step S12, the contents of the other registers saved in the stack are returned, the context switch is performed for the thread, and the CPU usage right is given. In step S13, the thread is executed.

In step S14, when an event that suspends or suspends the execution of the thread occurs, the registers are saved in the stack and the context switch to the OS is performed.
In step S15, the value of the stack base register (sbr) is saved in the thread control table, and
Update the thread control table such as thread execution time.

As described above, the present system is provided with a dedicated base register for making access to the stack at a relative address in the CPU, a group of computers connected on the network, or a parallel computer. When using the (virtual) shared memory mechanism, create a relative address space dedicated to the stack in a local memory space that is not the target of the (virtual) shared memory, divide the address space, and use a dedicated base register for part By creating the relative address space, it becomes possible to effectively use the local memory space in the system using the shared memory or the virtual shared memory mechanism.

[0061]

As described above, according to the present invention,
Since the stack space can be handled independently among the processors (CPUs), if threads are appropriately distributed to each processor, the total number of processors will be the maximum number of threads possible when the stack space is (virtually) shared. Threads close to the number multiplied by can be moved, enabling larger-scale calculations. Also, the size of the stack per thread can be made larger, and large data can be placed on the stack.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration example of the present invention.

FIG. 2 is a diagram illustrating address processing of a CPU according to the present invention.

FIG. 3 is a diagram showing an embodiment of the present invention.

FIG. 4 is a processing flowchart of a thread scheduler.

FIG. 5 is a diagram illustrating a problem of the conventional technique.

[Explanation of symbols]

 10 network 11, 11 'processor 12, 12' CPU address / logical address conversion circuit 13a-13d thread 14 space discrimination circuit 15 base register 16 adder circuit 17 address register

Claims (2)

[Claims] 1. In a system in which a plurality of processes or threads, which are the subjects of computation to obtain processor usage rights on a plurality of processors, operate, the address space of the memory is
A first address space shared by multiple processes or threads and a second address space not shared between processes or threads
And a memory from a process or thread, each of the processors holding a base address for making all or a specific portion of the second address space into a relative address space. Means for determining whether the access request is for all or a particular portion of the second address space, and if the access request is for all or a particular portion of the second address space, An address space sharing system, comprising means for adding the base address to an address and converting the address. 2. The address space sharing system according to claim 1, wherein all or a specific part of the second address space is a stack space used by each process or thread as a stack. system.