JP2000020327A - Device and method for distributed processing and network system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a distributed processing environment capable of efficiently performing the desired processing by efficiently executing message communication. SOLUTION: A transmission thread 820 of a process 810 divides the message of a transmission object into packets and transmits them to a process 850. In the process 850, the packets are parallelly received by plural reception threads and stored in a common area 883, and when the message is completed, it is dispatched to a high-order object. Its return value is dispatched to a reception thread 870 and transmitted to the process 810. In a reception thread 830 of the process 810, the message is constituted while similarly using a shared area 843 and when the message is completed, the reception of the return value is reported to the transmission thread 820 while using inter-thread communication or a message queue. Thus, the return value message is read out of the shared area 843 and dispatched to a high-order object by the transmission thread 820.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a distributed processing apparatus and method in which objects arranged on a plurality of nodes cooperate to perform desired processing while efficiently performing message communication, and a method for such distributed processing. The present invention relates to a network system that provides a processing environment.

[0002]

2. Description of the Related Art A network system in which various computing resources are connected via a network such as a telephone line, a cable television, a satellite communication, or the like, is rapidly spreading. Further, in such a network system, there is a growing expectation for parallel distributed processing in which a plurality of objects are arranged in a network wide manner and linked to perform desired processing. According to such distributed processing, it becomes possible to efficiently use computation resources on the network to perform desired processing at high speed, and to effectively use contents on the network.

[0003] In order to perform such distributed processing, each object communicates via a network.
It is fundamental to operate in cooperation. As a communication method, RPC (Remote Procedure Call) is often used. By the way, in order to perform such distributed processing efficiently, it is necessary to efficiently communicate between each object, in other words, between each process that provides an execution environment for the object at each node on the network. is there. That is,
It is desired that messages can be simultaneously communicated between processes of an unspecified number of nodes.

[0004]

However, in the past, one process is usually provided with one receiving thread, so that the transmitted message is often taken out. Therefore, if the speed of the accumulation in the socket queue is high, there is a problem that the retrieval cannot keep up, the socket queue is saturated, and the message cannot be properly received. Also,
Even in the case where a plurality of receiving threads are provided, each receiving thread is associated with a message and receives only a packet constituting a target message.
Therefore, the speed of retrieving a packet from the socket queue is not much different from the case of a single receiving thread.
As described above, there is a problem that the socket queue may be saturated and the message may not be properly received.

In the current inter-process communication using the RPC as it is, communication is performed by a method in which a transmitting thread transmits a request as a client and receives a response returned from a server partner on the same thread. I have. However, in terms of the relationship between the distributed objects on the network, it is desirable to perform communication in an equal relationship, not in a client-server relationship.
Then, it is preferable that each node appropriately transmits a large number of requests to an unspecified large number of opponents and appropriately processes responses sequentially returned. There is a demand for realizing such a framework.

[0006] Further, in such a network system, it is desirable to provide functions without being restricted by a programming language or the like. However, at present, when the code using the message passing mechanism on the distributed processing system and the code providing the message passing mechanism are described in different languages,
In some cases, the thread cannot be controlled from a higher-level application. Further, between programs written in different languages, for example, it is sometimes impossible to create a process, create a thread, or perform communication between threads. Therefore, there is also a problem that it is more difficult to construct a suitable environment related to message communication as described above.

Accordingly, it is an object of the present invention to enable efficient communication of a large amount of messages between processes having objects described in an arbitrary language, whereby the objects cooperate to efficiently execute desired processing. An object of the present invention is to provide a distributed processing device that can be executed well. Another object of the present invention is to enable efficient communication of a large amount of messages between processes having objects described in an arbitrary language, whereby the objects cooperate to efficiently execute desired processing. An object of the present invention is to provide a distributed processing method that can be executed. Another object of the present invention is to enable efficient communication of a large amount of messages between processes connected via a network, and for objects described in an arbitrary language distributed and arranged in the processes. It is an object of the present invention to provide a network system capable of providing a distributed processing environment capable of efficiently executing desired processing as a whole.

[0008]

SUMMARY OF THE INVENTION Accordingly, a distributed processing apparatus according to the present invention is a distributed processing apparatus in which a plurality of objects cooperate to perform a desired process. One or more each formed on a node, a plurality of processes that provide an execution environment for the arranged objects, and the plurality of objects each arranged in any of the plurality of processes, Each of the processes divides a message to be transmitted based on the operation of the object arranged in the process into a plurality of packets, and a transmission module that transmits the packet to a desired process, and receives the transmitted packets, respectively. A plurality of receiving modules for storing the received packet in a predetermined shared storage area accessible from any processing module in the process. And when all packets of one message are stored in the shared area, the processing module that substantially reads the message and performs a predetermined process based on the operation of the object arranged in the process. A selection module to be provided.

A distributed processing method according to the present invention is a distributed processing method in which a plurality of objects cooperate to perform a desired process. One or more processes for providing an execution environment are arranged, and the plurality of objects are
Each of the plurality of divided packets is arranged in any one of the plurality of processes, and a communication target message generated based on the operation of each of the objects arranged in the process is divided into a plurality of packets. Is transmitted by a transmission module in the process, and in each of the processes in which the respective packets are transmitted, a plurality of reception modules in the process receive the transmitted packets in parallel, and When the message is stored in a predetermined shared storage area and all the packets of one message are stored in the shared area, the message is substantially read out, and the read message is read by the operation of the object arranged in the process. To a predetermined processing module that performs predetermined processing based on the plurality of objects, It performed in cooperation desired processing by communicating a message of the desired content to each other through the process.

[0010] The network system of the present invention comprises:
A plurality of processes each formed on the node, wherein a plurality of nodes connected via a network and a plurality of objects for performing desired processing in cooperation with each other are arranged, A transmission module that divides a message to be transmitted based on the operation of the object arranged in the process into a plurality of packets, and transmits the packets to a desired process via the network; and A plurality of receiving modules for receiving the received packet and storing the received packet in a predetermined shared storage area accessible from any processing module in the process, and storing all the packets of one message in the shared area. If the message is read, the message is substantially read and the object placed in the process is read. And a plurality of processors and a selection module for providing a processing module for performing a predetermined processing based on the bets operation.

Preferably, the plurality of receiving modules include:
Each packet of a plurality of messages transmitted to the process is sequentially received in parallel without being associated with each message, and the received packets are stored in the shared storage areas. Also preferably, each of the plurality of receiving modules is configured to identify each of the received packets based on identification information of a message included in the packet and information indicating a position of the packet in the message. Then, it is stored in a predetermined area of the shared storage area. Preferably, when the content of the message received and stored in the shared area is a reply content to a message sent earlier than the process to another process, the transmission module that transmitted the original message However, as the selection module, the stored message is read and provided to a processing module that performs the predetermined processing.

[0012] More specifically, each of the plurality of receiving modules is configured so that the packet received and stored in the shared storage area is the last packet of a message included in the packet. Is a response to a message already sent from the process to another process, notifies the transmitting module that transmitted the original message that the message having the response has been received.

[0013] More specifically, each of the plurality of receiving modules is configured so that the packet received and stored in the shared storage area is the last packet of the message that the packet comprises, If the message is a reply to a message that has already been sent from the process to another process, the data indicating that the message having the reply has been received can be referred to from the transmitting module that transmitted the original message. The transmission module monitors arrival of a message having the reply content by monitoring the predetermined queue, and reads the message from the shared storage area.

[0014] More specifically, the transmission module detects arrival of the message having the reply content by monitoring messages sequentially formed in the shared storage area by the plurality of reception modules, and The message is read from the storage area.

Preferably, when the content of the message received and stored in the shared area is a new message which is not a reply content to a message already sent from the process to another process, the message is Is a selection module that reads out a message, and reads out the stored message and provides it to a processing module that performs the predetermined processing. Specifically, the communication of the message is a remote procedure call (RP).
C: Remote Procedure Call)).

[0016]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described.
This will be described with reference to FIGS. In the present embodiment, a network system in which a plurality of nodes including a computer device and the like are connected via a network, wherein each node cooperates while communicating via the network to perform a desired calculation process. The present invention will be described by exemplifying a network system that provides a distributed processing environment for performing the above.

First, the overall configuration of such a network system will be described. FIG. 1 is a diagram schematically showing the overall configuration of the network system. The network system according to the present embodiment includes the Internet, C
In a network environment composed of various communication media such as ATV network and satellite communication, distributed resources on the network are logically regarded as one integrated computing environment.
It provides a distributed processing environment. Here, the distributed resource provides an arbitrary calculation function such as a database, a high-performance calculation server, and an information source of a WWW page, in addition to the hardware function of each node. Also,
The calculation includes execution of a wide range of various processes.

On such a distributed processing environment, distributed modules (sometimes referred to as concurrent objects), which are computational entities, are arranged to form a computation space. A plurality of calculation spaces can be individually formed on the same network environment. This distributed module is, specifically, a distributed object that can be remotely called, a distributed function, or a shared variable that can be referred to via a network (hereinafter, referred to as a global shared variable).
With these, each calculation of distributed object-oriented calculation, distributed function (procedure) call, and global shared variable processing is performed in parallel.

Communication between concurrent objects such as calling distributed object methods and distributed functions and accessing global shared variables for performing such calculations is as follows:
Asynchronous message transmission (one-way communication without return value),
It is performed by three types of methods: a delay evaluation type synchronous call (blocks when a return value is accessed) and a complete synchronous call (the caller blocks until a return value arrives). Note that these communications are collectively referred to as message communications. That is, such a network system can be said to be a system in which concurrent objects distributed and arranged on the network sequentially execute parallel computations by message communication, and advance desired computations as a whole by chaining messages.

As shown in FIG. 2, each concurrent object arranged in the computation space generates and manages a process in a network node. A process is a computing environment for concurrent objects that provides a mechanism for sending and receiving messages between concurrent objects. Therefore, it is provided with an application language interface (API) and a system interface to the OS and hardware, and can operate according to the difference in the execution environment such as the programming language, OS, hardware, and network. The message can be transmitted between different languages, and an arbitrary data structure can be handled as an argument. As a result, an execution environment obtained as a set of such processes functions as a common virtual computing environment in the global space.

Next, a description will be given of a message transmitting / receiving mechanism, which is a mechanism for performing a desired distributed processing performed between such processes. FIG. 3 is a diagram for explaining a message transmitting / receiving mechanism provided by the process as described above. The process 100 includes, as components related to message transmission / reception, an application programming function 120, a scheduler 130, a reference resolver 140, a message communication unit 150, an operating system interface (OSI / F) 160, and a communication media interface (communication media I / F).
F) It has 170.

Application programming function 12
0 indicates to the concurrent object 110 an API (Application Programming Interface) related to message transmission.
ace). The scheduler 130 controls calculations in the process, such as executing the received message. The reference resolver 140 refers to the reference table 141 and specifies a calculation entity that performs message transmission. The message communication unit 150 is a function for transmitting and receiving a message, and includes a message sender 151 and a message dispatcher 152.

The OSI / F 160 is a system programming interface for performing system-dependent processing while ensuring portability and portability. The communication media I / F 170 is a system programming interface that interfaces with a communication medium, which is necessary for each concurrent object to perform message communication. The communication media I / F 170 according to the present invention will be described later in detail.

FIG. 4 shows the flow of the message transmission process in the process 100 having such a configuration. In the message transmission, by calling a message transmission function on the API of the application programming function 120, the structure of the argument is recognized and converted into a logical form that can be transmitted, and the reference resolver 140 resolves the reference of the transmission destination object. The message communication unit 150 performs transmission processing. At this time, depending on the location of the transmission destination, the RPC between nodes via the communication media I / F 170, the IP within the node
C or the communication in the process is selected and transmitted.
The reason for using RPC between nodes is to grasp the arrival of a message. If the transmission destination is a domain or an object cluster, multicast is performed for each process. The communication medium I / F 170 is prepared for each corresponding medium, and can specify a medium to be used according to the type of data and the content of control.

FIG. 5 shows a flow of a message receiving process in the process 100 having such a configuration. When a message is received via the communication media I / F 170, the message dispatcher 152 specifies a receiving object from the received remote identifier, generates a function descriptor for performing the processing, and outputs the function descriptor to the application programming function 120. Message arguments are expanded from a logical format into a data structure. The function descriptor holds information for invoking an object method and a distributed function or access to a shared variable, a time attribute and a priority of a message, and information for calling a message transmission source object. This function descriptor is passed to the scheduler 130 for processing. When the message transmission is performed by synchronous calling, the evaluation value obtained by the execution by the scheduler 130 is similarly converted into a logical form, and transmitted to the calling side by the same message sender 151. If the reference is relayed, a new global reference is obtained from the remote identifier, so that the message dispatcher 152 calls the message sender 151 as it is.

Next, in such a network system and message communication system, a communication media I / F 170 that actually performs message communication in response to a message communication request from a host will be described in detail. Here, a case will be described in which message communication is performed between nodes via a communication medium according to the present invention.

FIG. 6 shows the communication media I / F1 of the process.
70 is a diagram showing the logical configuration of a node 70 and the flow of processing when message communication is performed between the communication media I / F 170a of the process on the node A and the communication media I / F 170b of the process on the node B. As illustrated, the communication media I / F 170 has a communication method dispatcher 171 and a communication unit for each communication method.

In response to a message communication request input from the message sender 151 of the message communication unit 150, the communication method dispatcher 171 selects a communication medium for performing the message communication and sends the communication medium to the communication unit of the communication medium. Enter information related to message communication. The communication unit is provided with various processing units corresponding to communication media such as a CATV network and a telephone line network. As one of them, an RPC unit 172 for performing message communication by RPC via the Internet is provided. Have been. In the present embodiment, hereinafter, this RPC unit 1
A communication control method using RPC 72 will be described.

When the RPC is used, a communication error is automatically retried,
The use of receiving a CK can improve the reliability of transmission and reception. In addition, communication media I / F1
The conversion of the management information added by 70 to the message body, such as the byte order, can also be performed using a filter generated by the protocol compiler of the RPC. However, the conversion relating to the message body is performed by the application programming function 120. Note that the RPC of the present embodiment is, specifically, Solaris (Sun, USA).
ONC (Op) in Microsystems, Inc.
en Network Computing) RPC, Windows NT (M
ONC-RPC fo in the registered trademark of icrosoft Corporation)
r Windows NT.

The RPC section 172 uses TCP or UDP as a transport layer protocol. When TCP is selected as the transport layer protocol, reliability can be improved by the TCP service. However, since TCP is connection-oriented, there is a problem in performance when many connections are established at the same time. If that poses a problem, UDP may be selected as the transport layer protocol. When UDP is selected, the reliability like TCP cannot be expected, but the performance is good because there is no service for improving the reliability. In the following description, a case where UDP is selected will be described.

As shown, the RPC section 172 further has a message handler 174 and a communication manager 173.

The message handler 174 performs message division and restoration processing. UDP has a limit on the amount of transmission data. Therefore, when UDP is used as the transport layer protocol and a message exceeding the transmission amount limit is transmitted, the message is divided into packets of a fixed data size, transmitted, and restored on the receiving side. The message handler 174 performs such division and restoration processing. As illustrated, the message handler 174 includes a message fragmentation unit (Message Fragmentati
on) 175, Message Reassembl
age) 176 and Buffer Manager
177.

The message division unit 175 divides the message into packets of a fixed size and outputs the packets to the communication manager 173. At this time, the message division unit 175 adds information necessary for restoring the message on the receiving side to each of the divided packets. The information to be added is the sender's IP address and process ID, message length, message sequence number, total number of packets, and packet sequence number. The message sequence number is a number indicating the order of the message transmitted in the process, the total number of packets is a number indicating how many packets the message has been divided into, and the packet sequence number is the number of the divided message. Is a number indicating the order of the packet.

Message reassembly section 176 assembles the received packet input from communication manager 173 into an original message, and outputs the original message to message dispatcher 152 of message communication section 150. At this time, the transmitting side only transmits the divided packets in order, but the receiving side may receive a packet of another message between packets of one message. Therefore, the message reassembly unit 176 has a reception buffer for each message, distributes the received packet to the corresponding buffer, and assembles a message.

The buffer manager 177 manages a buffer (not shown) in response to a request from the message reassembly unit 176. The buffer manager 177 generates a buffer when a new buffer is generated from the message reassembly unit 176. In addition, the sequentially input packets are sequentially stored in a buffer corresponding to a message included in the packet. When all the packets constituting one message are stored and the message is completed, the message is read from the buffer and output to the message reassembly unit 176.

The operation of reassembling a message by the message reassembly unit 176 and the buffer manager 177 will be specifically described. If the received packet is the first packet of the message, the message reassembly unit 176 requests the buffer manager 177 for a buffer corresponding to the message length.
The buffer is identified by the sender's IP address, process ID, and message sequence number. Then, the received packet is stored in the buffer generated based on the request. When the received packet is the second or subsequent packet of the message, the buffer searching unit 11 searches for a buffer having the same IP address, process ID, and message sequence. Then, the received packet is stored in the buffer. Then, when the stored packet is the last packet of the message, the restored message is read from the buffer and output to the message communication unit 150.

Next, the communication manager 17
3 will be described. Communication Manager 1
Reference numeral 73 actually transfers a packet constituting a message to / from a communication manager 173 of a process on a desired node via a network. The function of the communication media I / F 170 is to provide a message transmission / reception function, and it is important to efficiently cope with a large number of transmission / receptions that occur simultaneously. In the communication media I / F 170 of the present embodiment, the client handle and the socket used under the RPC are used to
An efficient communication function is realized, and the communication media I / F 170 controls the actual client handle and socket.

The client handle is a handle required when transmitting a message by RPC. In the communication manager 173, the client handle is given for each connection between the processes, and is managed in the transmitting process. That is, a socket descriptor described later, a socket descriptor of a process that is a communication partner, and the process are uniquely associated with the client handle. Therefore, each sending thread in a process can send a message to a target process only by selecting an appropriate client handle. When the destination process is the same as the previously transmitted process, the previously established connection is reused, and the reconnection is unnecessary.

With regard to sockets, the number of socket descriptors that can be used in a process is limited. Therefore, assuming that there is a large number of simultaneous transmissions and receptions, the socket descriptors must be efficiently allocated and used. Therefore, the communication manager 173 assigns a socket descriptor to the connection for each connection on each of the transmission side and the reception side. At this time, the communication manager 1 of the transmitting process which is the RPC client
73a allocates a socket descriptor at the time of a connection request with the RPC server. In addition, the communication manager 173b of the RPC server, which is the receiving process, receives the default socket descriptor at the time of the first reception, and immediately allocates another socket descriptor for communication with the process at the time of the first reception. . At this time, in consideration of the load on each socket of the receiving process, the socket descriptor to be allocated is determined so that the load is distributed.

Therefore, the initial communication is performed between the assigned socket on the transmitting side and the default socket on the receiving side. However, in the subsequent communication, the load on the assigned socket on the transmitting side and the load on the receiving side are set. This is done with the assigned socket taking into account. If the number of connections in each process is the number of socket descriptors within the specified limit in the allocation of each socket descriptor on the transmitting side and the receiving side, the allocation is performed by the RPC function. If it exceeds, the socket descriptors assigned to the RPCs are assigned one after another. Thus, in such a case, multiple client handles will share the socket descriptor.

Under the RPC, a combination of a server host, a program number, a version number and a protocol is used to specify a connection destination. In order to specify a socket to be connected, a version number is assigned to each socket and the version number is specified.

The communication manager 17
3 registers the generated socket in the ONC-RPC port mapper 180. ONC-RPC port mapper 1
When the message is registered in the RPC 80, when a message reaches the socket used by the RPC, a notification is sent to the RPC unit 172, and the process can be shifted.

The communication media I / F 17 having such a configuration is described.
The flow of processing when message communication is performed via a network between processes each having 0 will be described. Note that here, processing is performed when a message that requires a return value is transmitted from a concurrent object running on a process on node A to a parallel object running on a process on node B. Will be described. First, a message transmission request is made to the message communication unit 150a of the process of the node A, and the message dispatcher 152a of the message communication unit 150a includes the message to be transmitted, the IP address of the transmission destination, and the process ID. The information related to the message is transmitted to the communication media I / F 170a, and a message transmission is instructed.

In the communication medium I / F 170a, a communication medium is selected in the communication method dispatcher 171a. For example, when performing communication via the Internet, the message division unit 17 of the RPC unit 172a
In step 5a, the message is divided into a plurality of data volumes having a data volume suitable for transmission by UDP, and a transmission packet is generated. Then, the packet is transmitted to the desired process of the node B by the request transmission function of the communication manager 173a.

This packet is appropriately received by the request receiving function of the communication manager 173b of the communication media I / F 170b of the process of the node B, and is received by the message reassembly unit 1 of the message handler 174b.
At 76b, the buffer manager 17 is temporarily
7b is assembled as a message. The assembled message is passed to the message dispatcher 152b of the message communication unit 150b, and is executed after being scheduled and managed by a scheduler 400b (not shown).

When the execution is completed, a message for returning a return value such as a result is transmitted from the message dispatcher 152b of the message communication unit 150b to the R.
Instructed by the PC unit 172b. Communication media I / F17
0b, the communication medium I /
Similarly to the packet generation processing in F170a, the message division unit 175b generates a plurality of packets based on the return values, and transmits the packets to the node A process by the return value transmission function of the communication manager 173b.

This packet is received by the return value receiving function of the communication manager 173a of the communication media I / F 170a of the process of the node A, and the message is processed in the same manner as the message restoration process in the communication media I / F 170b of the node B described above. Handler 174a
The message reassembly unit 176a temporarily assembles the message by temporarily storing it in the buffer manager 177b. The assembled message, that is, the return value is transmitted to the message communication unit 150a.
To the message sender 151a, and is returned to the message transmission source concurrent object or the like.

Next, logically, the process 100 having such functions and its communication media I / F 170
Will be described with reference to FIG. 7 when referring to FIG. 7, in the case where is actually realized in a process in a multi-thread environment. FIG. 7 is a diagram showing a thread configuration and a flow of processing when a message is communicated between the remote function calling process on the node A and the remote function calling process on the node B.

As shown in the figure, each process has a plurality of transmission threads and a plurality of reception threads, and the transmission and reception of the packets constituting the message are performed in parallel. The transmission threads 820 and 870 are threads for transmitting a message sent from a higher order to the other party, and are generated each time a message is input.
The receiving threads 830 and 860 are threads for receiving the result of sending a message, and are provided in a plurality of processes. As described above, there are a plurality of transmission threads and a plurality of reception threads in each process.
In the following description, the transmission thread 820 and the reception thread 830 for the transmission side (the process 810 of the node A), and the reception thread 860 and the transmission thread 870 for the reception side (the process 850 of the node B) are concrete examples. The description is made as follows.

The flow of processing in each thread of each process will be described below. In the transmission thread 820 of the transmission-side process 810, a function call from the application programming function 120 causes a message sender 821 as a user of the communication media I / F 170 (the message sender 151 of the message communication unit 150 in FIG. 3) to call the function. When a message request is received, the CMI sender 822, which is the highest function of the communication media I / F 170, receives the message request and passes it to the message request transmission function 823. The message request transmission function 823 is
The information of the message transmission destination is passed to the handle manager 824. The handle manager 824 acquires the information of the RPC client handle to be used by referring to the client handle management table 840 based on the information, and returns the information to the message request transmission function 823. The message request transmission function 823 transmits a message via the client handle and the port 841 associated with the client handle.

At this time, when the communication is performed for the first time with respect to the destination process, the default port 885 (configured in correspondence with the socket descriptor and the client handle) of the destination process is used. To send a message. As described later, the receiving thread 860 of the receiving process 850, upon receiving a packet on the default port 885, executes the process 8 of the transmission source.
And assigns a new port 886 to port 10 and sends information indicating the port to the ACK for confirming reception of the packet.
At the same time, it notifies the transmitting process 810. Therefore, the transmission thread 820 registers the information of the port 886 in the client handle management table 840 via the handle manager 824. Thereafter, communication with the process is performed via the notified port. Therefore, the transmission thread 820 also sequentially transmits the remaining packets of the message to be transmitted to the port 8
86.

When the transmission of the message is completed, if the message is a message that does not require a return value,
The transmission thread 820 ends the processing. If the message requires a return value, the sending thread 820 waits for the return value. The return value indicates whether the receiving thread 830 in the transmitting process 810 that has received the notification notifies the transmitting thread 820 by inter-thread communication,
In some cases, a notification to that effect is sent to a message queue (not shown).

In each case, the transmission thread 820 C
The MI receiver 825 receives this notification and
The return message stored in 43 is read and returned to a higher-level function via the message sender 821. Note that this notification also includes address information indicating where in the shared area the message is located. Therefore,
By referring to the address information, the transmission thread can retrieve a desired message without searching for a completed message.

Each of the plurality of receiving threads of the receiving process 850 monitors the default port 885 of the receiving process 850 by a system call (select) for monitoring a file descriptor. Then, the receiving thread that first detects the message that has reached the default port 852 (this is referred to as a receiving thread 860) receives the message. The receiving thread 860 that has received the first message allocates a new port 886 for receiving the message to the transmitting process 810 that has transmitted the message, as described above. Then, the information of this port is transmitted to the arrival confirmation A
Along with CK, the transmission thread 820 is notified. As a result, the subsequent message transmission from the transmitting process 810 is performed to the port 852.

The receiving thread 860 registers the message collection service by the service registration function 862 at the generation stage, so that the collection service 863 for collecting the packet addressed to the receiving thread 860 is performed. Therefore, subsequent collection of packets is performed by the collection service 863.

In the receiving thread 860, the message request receiving function 861 switches the ports 885 and 88
6 and receives it through the collection service 86
3 are sequentially stored in the shared area 883. When one message is assembled in the shared area 883, if the message is not the return value of the previously transmitted message, that is, if the message is a request of a procedure or the like to the receiving process 850, the CMI receiving function 864 converts this message. The data is read and output to the application programming function 120 via the message dispatcher 865 (corresponding to the message dispatcher 152 of the message communication unit 150 in FIG. 3). If the message assembled in the shared area 883 is the return value of the previously transmitted message, the collection service 863 performs the inter-thread communication to execute the receiving process 8 waiting for the return message.
The receiving thread in 50 is notified that the return message has been received.

The receiving thread 87 of the receiving process 850
0 is also the sending thread 8 of the sending process 810 described above.
It has exactly the same function as 20. A description will be given of the processing flow in the case of returning a return value for a message transmitted from the transmission side process 810. A request for a return value message is received via the message sender 871 by a function call from the application programming function 120, and the CMI sender is returned. 872 further receives it and passes it to the message return value transmission function 873. The message request transmission function 823 refers to the client handle management table 880 via the handle manager 874 to obtain information on the RPC client handle used for transmitting to the process 810, and based on the information, obtains the port 881 Send a return message to the process 810 via.

At this time, if the transmission of this message is the first communication from the process 850 to the process 810, the process 810 allocates a port from a default port to a specific port. Is the same as in the case of the process 850 described above, and a description thereof will be omitted. Then, from process 850 to process 8
When transmission of the return message to port 10 via ports 881 and 845 is completed, transmission thread 870 terminates the process.

The receiving thread 83 of the transmitting process 810
0 is the receiving thread 8 of the receiving process 850 described above.
It has exactly the same function as 60. The process flow when a return message is transmitted from the receiving process 850 will be described. Among the plurality of receiving threads of the receiving process 850, the receiving thread 830 that first detects the arrival of the message receives the message return value. The function 831 sequentially receives the return message packet via the port 845, and the collection service 833 registered in advance by the service registration function 832 sequentially stores the packet in the shared area 843. When the return message is assembled in the shared area 843, the collection service 863 notifies the transmission thread 820 waiting for the return message of the reception of the return message by inter-thread communication.

As described above, in the communication media I / F of the present embodiment, the message transmission / reception is performed in parallel by the multithread, so that the performance of the message communication can be improved. In particular, in the case of UDP, since a communication amount is limited, one message may be divided into a plurality of packets and transmitted. In such a case, a plurality of packets of one message may be transmitted by a plurality of threads. Can be received efficiently.

Further, in the present embodiment, the notification of the arrival of the return message can be performed between the threads via the message queue, so that the user program and the communication medium I / F 170 are written in different languages. Thus, even when the so-called inter-thread communication cannot be performed, it is possible to appropriately notify, for example, that the return value has been received. Therefore, it is possible to provide a distributed processing environment having a high degree of freedom without any restriction on the description language of the application program. Further, by using the message queue in this way, a processing module waiting for the arrival of a return message can wait only for a message required by itself, thereby omitting an operation such as a search.

The present invention is not limited to the present embodiment, and various modifications are possible. For example, in the present embodiment, a method using inter-thread communication and a message queue has been described as a method of detecting that a return value has been transmitted by a transmission thread that has transmitted a message. But, for example, if the sending thread waits for a return value,
The arrival of the return value may be detected by monitoring the shared storage area where the received message is assembled and detecting the storage of the return value message. The configuration of the threads in each process, the functions in each thread, the configuration of the shared area, the configuration of the ports, and the like may be arbitrarily determined.

[0063]

As described above, according to the distributed processing apparatus and method of the present invention, a large amount of messages can be efficiently communicated between processes having objects described in an arbitrary language. As a result, the objects can cooperate to execute the desired processing efficiently. Further, according to the network system of the present invention, a large amount of messages can be efficiently communicated between processes connected via a network, and objects described in an arbitrary language distributed and arranged in the processes can be used. Thus, it is possible to provide a distributed processing environment that can efficiently execute desired processing as a whole.

[Brief description of the drawings]

FIG. 1 is a diagram schematically illustrating an overall configuration of a network system according to an embodiment of the present invention.

FIG. 2 is a diagram illustrating a relationship between concurrent objects and processes arranged in a computation space illustrated in FIG. 1;

FIG. 3 is a diagram for explaining a message transmission / reception mechanism provided by the process shown in FIG. 2;

FIG. 4 is a diagram illustrating a flow of a message transmitting process by the message transmitting / receiving mechanism illustrated in FIG. 3;

FIG. 5 is a diagram illustrating a flow of a message receiving process by the message transmitting / receiving mechanism illustrated in FIG. 3;

6 is a diagram illustrating a logical configuration of a communication medium I / F illustrated in FIG. 3 and a flow of a process when message communication is performed between nodes;

FIG. 7 is a diagram illustrating a configuration of a thread and a flow of processing when a message is communicated between processes on different nodes;

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 100 ... Process 110 ... Concurrent object 120 ... Application programming function 130 ... Scheduler 140 ... Reference resolver 141 ... Reference table 150 ... Message communication unit 151 ... Message sender 152 ... Message dispatcher 160 ... OSI / F 170 ... Communication media I / F 180 ... ONC-RPC port mapper 810,850 ... Process 820,870 ... Transmission thread 821,871 ... Message sender 822,872 ... CMI sender 823,873 ... Message transmission function 824,874 ... Handle manager 825,875 ... CMI receiver 830, 860: Receiving thread 831, 861 Message receiving function 832, 862 Service registration function 833, 863 Collection service 834, 864 CM Receiving function 835,865 ... message dispatcher 840,880 ... client handle management table 841,845,881,885,886 ... port 843,883 ... the shared area

[Procedure amendment]

[Submission date] June 21, 1999 (1999.6.2
1)

[Procedure amendment 1]

[Document name to be amended] Statement

[Correction target item name] Claims

[Correction method] Change

[Correction contents]

[Claims]

[Procedure amendment 2]

[Document name to be amended] Statement

[Correction target item name] 0008

[Correction method] Change

[Correction contents]

[0008]

Therefore, a distributed processing apparatus according to the present invention provides a desired processing in which a plurality of objects cooperate with each other.
Distributed processing apparatus for performing
A node and one or more each formed on said node
That provide an execution environment for the placed objects
Process and each of the plurality of processes.
And the plurality of objects to be placed,
Each of the processes is an object located in the process.
Multiple messages to be sent based on the
A transmission module that divides packets and sends them to the desired process
Module and any processing module in the process
It is provided in an accessible shared storage area,
Storage of the packets that make up the message for each page
Receive buffer having a storage area in which areas are arranged in order
And access from any processing module in the process.
Message queues that can be
Receive packets in any order of the message
It is stored in a predetermined storage area of the reception buffer, and the received
The received packet is the
If the packet was stored later,
Information indicating the location of the message
A plurality of receiving modules for writing to the queue, and the message
Reads the information of the Sage Queue, indicated by the information
The message stored in the receive buffer
Read out the object and place the object in the process.
Processing module that performs predetermined processing based on the operation of the project
And a selection module to be provided.

[Procedure amendment 3]

[Document name to be amended] Statement

[Correction target item name] 0009

[Correction method] Change

[Correction contents]

Preferably , each of the plurality of receiving modules is
Each of the received packets has the packet
And the identification information of the message
Information indicating the position of the packet in the message.
And stores it in a predetermined area of the reception buffer. Also
Preferably, said received and stored in said receive buffer
The content of the message is passed from the process to another process.
If the response is to a message already sent,
The sending module that sent the original message is
As the selection module, the stored message
Read and provide to the processing module that performs the predetermined processing
I do.

[Procedure amendment 4]

[Document name to be amended] Statement

[Correction target item name] 0010

[Correction method] Change

[Correction contents]

Preferably, the plurality of receiving modules are provided.
Are the packets received and stored in the reception buffer.
Packet is the last packet of the message that constitutes the packet.
Packet, and the message is
Process to messages already sent by the process to other processes.
The original message was sent if the reply was
A message having the reply content to the sending module
Notify that the message has been received. Also preferably, the transmission mode
Joule is received by the plurality of receiving modules.
To monitor messages that are sequentially formed in the buffer
Detects the arrival of the message having the reply content
Then, the message is read from the reception buffer.
Also preferably, the received data is stored in the reception buffer.
The content of the received message is
Is not a reply to a message already sent to
If it is a new message, receive the message
Said receiving module, said stored message
To the processing module that performs the predetermined processing.
Offer.

[Procedure amendment 5]

[Document name to be amended] Statement

[Correction target item name] 0011

[Correction method] Change

[Correction contents]

Further , the distributed processing method of the present invention includes a plurality of
A distributed processing method in which objects cooperate to perform desired processing
At each of a plurality of interconnected nodes
Processes that provide an execution environment for
One or more objects, and the plurality of objects are
Each of which is located in one of the plurality of processes,
Based on the behavior of each of the above objects
Each generated message to be communicated
And each of the plurality of divided packets is
The packet is transmitted by the transmission module in the
In each of the processes to which the
A plurality of receiving modules for transmitting the transmitted packet.
Received in parallel, and the received packet is
Predetermined reception accessible from any processing module
The message is provided in the buffer for each message.
Storage where the storage areas of the packets that make up the page are arranged in order
Stored in a receive buffer having an
When all packets are stored in the reception buffer
Information indicating the position of the message in the reception buffer.
Information to the message queue, the message
Stored in the receive buffer based on the contents of the queue
Substantially reading the message
The object message placed in the process.
A predetermined processing module that performs a predetermined process based on the operation of the
The plurality of objects in the process
Communication of messages with desired contents via
And perform the desired processing in cooperation.

[Procedure amendment 6]

[Document name to be amended] Statement

[Correction target item name] 0012

[Correction method] Change

[Correction contents]

[0012] Preferably, the received packet is
Of the message that the packet has
Indicates the identification information and the position of the packet in the message.
Information in a predetermined area of the reception buffer based on the
It is memorized. Also preferably, it is stored in the reception buffer.
Message is sent prior to the process.
If the reply content corresponds to the message
The transmitting module that transmitted the message
Offer a sage.

[Procedure amendment 7]

[Document name to be amended] Statement

[Correction target item name] 0013

[Correction method] Change

[Correction contents]

Preferably, the plurality of receiving modules are provided.
Are the packets received and stored in the reception buffer.
Packet is the last packet of the message that constitutes the packet.
If it was a packet, the message was sent to the destination
Response to the message
The reply to the sending module that sent the message
Notify that a message having contents has been received. Also
Preferably, the transmitting module is the receiving module
The messages sequentially formed in the reception buffer by
By monitoring, a message having the reply content
Is detected, and the message is received from the reception buffer.
Is read.

[Procedure amendment 8]

[Document name to be amended] Statement

[Correction target item name] 0014

[Correction method] Change

[Correction contents]

Further , the network system of the present invention comprises:
Work with multiple nodes connected via a network
Multiple objects to perform desired processing
One or more are formed on each of the nodes
Multiple processes, each assigned to the process
Message to be sent based on the behavior of the object
Divided into multiple packets and sent through the network
And a transmission module for transmitting to a desired process.
Accessible from any processing module in the process
It is provided in a predetermined reception buffer, and
Storage areas of packets constituting the message are arranged in order
Receiving buffer having an allocated storage area and the process
Messages accessible from any processing module in the
Jqueue and each of the incoming messages
Receives packets in any order and places them in the receive buffer.
Stored in a fixed storage area and the received and stored packets
Was stored at the end of the message containing the packet
If it is a packet, the message in the receive buffer
Writes information indicating the position of the message to the message queue.
A plurality of receiving modules,
Read the information and receive the reception buffer indicated by the information.
Read the message stored in the file
The behavior of the object placed in the process
Selection provided to the processing module that performs the predetermined processing based on the
And a plurality of processors having a selection module.

[Procedure amendment 9]

[Document name to be amended] Statement

[Correction target item name] 0015

[Correction method] Change

[Correction contents]

[0015] Preferably, the plurality of receiving modules include:
Each of the multiple messages sent to the process
Packets without being associated with each message
Receive the packet in parallel and receive the received packet
Each is stored in a buffer. Specifically, the communication of the message is performed by a remote procedure call (RPC: Remote Procedure Call).

 ────────────────────────────────────────────────── ─── Continued on the front page (72) Inventor Hirotoshi Maekawa 3-37 Akasaka, Minato-ku, Tokyo Digital Vision Laboratories, Inc. F-term (reference) 5B045 BB31 BB42 GG01 5B089 AA20 AB01 AC07 AD01 AF01 CC11 5B098 AA10 GA04 GA05 GC16 GC19

Claims (25)

[Claims] 1. A distributed processing apparatus in which a plurality of objects cooperate to perform a desired process, a plurality of nodes connected to each other, and one or a plurality of objects respectively formed and arranged on the nodes. A plurality of processes that provide an execution environment for each of the plurality of processes, and the plurality of objects that are respectively arranged in any of the plurality of processes. Each of the plurality of processes is an operation of the object arranged in the process Splits the message to be sent based on the
A transmitting module for transmitting to a desired process; and a plurality of receiving modules each for receiving the transmitted packet and storing the received packet in a predetermined shared storage area accessible from any processing module in the process. And when all the packets of the message 1 are stored in the shared area, the message is substantially read out,
A selection module that provides a processing module that performs a predetermined process based on an operation of the object arranged in the process. 2. The receiving module according to claim 1, wherein each of the plurality of receiving modules sequentially receives a plurality of packets of the plurality of messages transmitted to the process without being associated with each of the messages. 2. The distributed processing apparatus according to claim 1, wherein each of the distributed processing apparatuses stores the information in a storage area. 3. The receiving module according to claim 1, wherein each of the plurality of receiving modules identifies each of the received packets based on identification information of a message included in the packet and information indicating a position of the packet in the message. And storing the data in a predetermined area of the shared storage area.
3. The distributed processing device according to 1. 4. If the content of the message received and stored in the shared area is a reply to a message already sent from the process to another process, the transmitting module that has transmitted the original message is 4. The distributed processing apparatus according to claim 1, wherein the stored message is read as the selection module and provided to a processing module that performs the predetermined processing. 5. The reception module according to claim 1, wherein the packet received and stored in the shared storage area is a last packet of a message included in the packet, and the message is transmitted to the reception module. 5. The distribution according to claim 4, wherein, in the case of a reply to a message already sent to another process, the transmitting module that transmitted the original message is notified that the message having the reply has been received. Processing equipment. 6. The reception module according to claim 1, wherein the packet received and stored in the shared storage area is the last packet of a message included in the packet, and the message is transmitted to the process module. In the case of a reply content to a message already sent to another process, data indicating that a message having the reply content has been received is put into a predetermined queue which can be referred to from the transmission module that transmitted the original message. The distributed processing apparatus according to claim 4, wherein the transmission module monitors the predetermined queue to detect arrival of a message having the reply content, and reads the message from the shared storage area. 7. The transmission module monitors arrival of a message having the reply content by monitoring messages sequentially formed in the shared storage area by the plurality of reception modules, and receives the message from the shared storage area. The distributed processing device according to claim 4, wherein the distributed processing device reads a message. 8. If the content of the message received and stored in the shared area is a new message that is not a reply to a message already sent from the process to another process, the message is received. The distributed processing device according to claim 1, wherein the receiving module reads the stored message and provides the read message to a processing module that performs the predetermined processing. 9. The communication of the message includes a remote procedure call (RPC: Remot).
The distributed processing apparatus according to any one of claims 1 to 8, wherein the processing is performed by e Procedure Call). 10. A distributed processing method in which a plurality of objects cooperate with each other to perform a desired process, wherein a plurality of processes for providing an execution environment for an object arranged in each of a plurality of interconnected nodes. Arranging one or more objects, arranging the plurality of objects in any of the plurality of processes, and sending a plurality of communication target messages generated based on the operation of the respective objects arranged in the process to a plurality of objects, respectively. Each of the plurality of divided packets is transmitted by a transmission module in the process, and in each of the processes in which each of the packets is transmitted,
A plurality of receiving modules in the process receive the transmitted packets in parallel, store the received packets in a predetermined shared storage area, and store all the packets of one message in the shared area. In this case, the message is substantially read, and the read message is provided to a predetermined processing module that performs a predetermined process based on an operation of the object arranged in the process, and the plurality of objects execute the process. A distributed processing method for performing desired processing in cooperation with each other by mutually communicating messages of desired contents via the same. 11. The received packet is stored in a predetermined area of the shared storage area based on identification information of a message included in the packet and information indicating a position of the packet in the message. The distributed processing method according to claim 10, wherein the distributed processing method is stored in a storage device. 12. When the content of a message stored in the shared area is a reply corresponding to a message sent before the process, the message is sent to the sending module that sent the original message. The distributed processing method according to claim 10 or 11, which is provided. 13. The receiving module according to claim 1, wherein the packet received and stored in the shared storage area is a last packet of a message included in the received packet, and the message is transmitted to the destination earlier. 13. The distributed processing method according to claim 12, wherein, in the case of a reply content to the received message, the transmitting module that transmitted the original message is notified that a message having the reply content has been received. 14. Each of the plurality of receiving modules, when the packet received and stored in the shared storage area is the last packet of a message included in the packet, the message is transmitted first. In the case of a reply to the message, the data indicating that the message having the reply has been received is put into a predetermined queue that can be referred to by the transmission module that transmitted the original message. 14. The distributed processing method according to claim 12, wherein the predetermined queue is monitored to detect arrival of a message having the reply content, and the message is read from the shared storage area. 15. The transmission module monitors a message sequentially formed in the shared storage area by the reception module, detects arrival of a message having the reply content, and transfers the message from the shared storage area. The distributed processing method according to any one of claims 12 to 14, wherein the read processing is performed. 16. The communication of the message includes a remote procedure call (RPC: Rem).
The distributed processing method according to any one of claims 10 to 15, which is performed by ote procedure call)). 17. A plurality of nodes each formed on the node, wherein a plurality of nodes connected via a network and a plurality of objects for performing desired processing in cooperation with each other are arranged. Processes, each of which divides a message to be transmitted based on an operation of the object arranged in the process into a plurality of packets,
A transmitting module for transmitting to a desired process via the network; and a predetermined sharing unit for receiving a packet transmitted via the network and accessing the received packet from any processing module in the process. A plurality of receiving modules to be stored in a storage area; and when all packets of one message are stored in the shared area, the message is read out substantially;
A network system comprising: a plurality of processors having a selection module that provides a processing module that performs a predetermined process based on an operation of the object arranged in the process. 18. The plurality of receiving modules sequentially receive respective packets of a plurality of messages transmitted to the process in parallel without being associated with the messages, and share the received packets. The network system according to claim 17, wherein the network system stores the information in a storage area. 19. Each of the plurality of receiving modules identifies each of the received packets based on identification information of a message included in the packet and information indicating a position of the packet in the message. The network system according to claim 17, wherein the information is stored in a predetermined area of the shared storage area. 20. When the content of the message received and stored in the shared area is a reply to a message sent to another process earlier than the process, the transmitting module that transmitted the original message. 20. The network system according to claim 17, wherein the network module reads out the stored message as the selection module and provides the message to a processing module that performs the predetermined processing. 21. Each of the plurality of receiving modules, when the packet received and stored in the shared storage area is the last packet of a message included in the packet, 21. The network according to claim 20, wherein, in the case of a reply to a message already sent to another process, the sending module that sent the original message is notified that a message having the reply has been received. system. 22. Each of the plurality of receiving modules, when the packet received and stored in the shared storage area is the last packet of a message included in the packet, In the case of a reply content to a message already sent to another process, data indicating that a message having the reply content has been received is put into a predetermined queue which can be referred to from the transmission module that transmitted the original message. 22. The network system according to claim 20, wherein the transmission module detects arrival of the message having the reply content by monitoring the predetermined queue, and reads the message from the shared storage area. 23. The transmission module detects arrival of a message having the reply content by monitoring messages sequentially formed in the shared storage area by the plurality of reception modules, and receives the message from the shared storage area. 23. The network system according to claim 20, wherein a message is read. 24. When the content of the received message stored in the shared area is a new message that is not a reply to a message already sent from the process to another process, the message is received. 24. The network system according to claim 17, wherein the receiving module reads the stored message and provides the message to a processing module that performs the predetermined process. 25. The communication of the message includes a remote procedure call (RPC: Rem).
The network system according to any one of claims 17 to 24, wherein the network system is performed by ote procedure call)).